-- Logs begin at Tue 2022-10-18 10:53:42 UTC, end at Tue 2022-10-18 12:00:57 UTC. -- Oct 18 11:26:07.969610 dv-cinder-815582-17-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 18 11:26:07.982805 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 18 11:26:07.984924 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 18 11:26:07 2022] *** Oct 18 11:26:07.984924 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 18 11:26:07.984924 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 18 11:26:07.984924 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: nodename: dv-cinder-815582-17-iscsi Oct 18 11:26:07.984924 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: machine: x86_64 Oct 18 11:26:07.984924 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: clock source: unix Oct 18 11:26:07.984924 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: pcre jit disabled Oct 18 11:26:07.984924 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: detected number of CPU cores: 4 Oct 18 11:26:07.984924 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: current working directory: / Oct 18 11:26:07.984924 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: detected binary path: /usr/bin/uwsgi-core Oct 18 11:26:07.985564 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: your processes number limit is 47680 Oct 18 11:26:07.985564 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: your memory page size is 4096 bytes Oct 18 11:26:07.985564 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: detected max file descriptor number: 2048 Oct 18 11:26:07.985564 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: lock engine: pthread robust mutexes Oct 18 11:26:07.985564 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: thunder lock: enabled Oct 18 11:26:07.985564 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 18 11:26:07.985564 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 18 11:26:08.025330 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: Python main interpreter initialized at 0x562b3f0039c0 Oct 18 11:26:08.025330 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: python threads support enabled Oct 18 11:26:08.025330 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: your server socket listen backlog is limited to 100 connections Oct 18 11:26:08.025330 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: your mercy for graceful operations on workers is 90 seconds Oct 18 11:26:08.025712 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: mapped 403077 bytes (393 KB) for 2 cores Oct 18 11:26:08.025791 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: *** Operational MODE: preforking *** Oct 18 11:26:08.026047 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: *** uWSGI is running in multiple interpreter mode *** Oct 18 11:26:08.026047 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: spawned uWSGI master process (pid: 81688) Oct 18 11:26:08.026259 dv-cinder-815582-17-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 18 11:26:08.027199 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: spawned uWSGI worker 1 (pid: 81700, cores: 1) Oct 18 11:26:08.027199 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: spawned uWSGI worker 2 (pid: 81701, cores: 1) Oct 18 11:26:08.028609 dv-cinder-815582-17-iscsi devstack@keystone.service[81688]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 18 11:26:12.327215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 18 11:26:12.327215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: from cryptography.utils import int_from_bytes Oct 18 11:26:12.328009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 18 11:26:12.328009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: from cryptography.utils import int_from_bytes Oct 18 11:26:12.328153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 18 11:26:12.328153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: from cryptography.utils import int_from_bytes Oct 18 11:26:12.328153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 18 11:26:12.328153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: from cryptography.utils import int_from_bytes Oct 18 11:26:12.379027 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 18 11:26:12.379240 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 18 11:26:12.379371 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 18 11:26:12.379474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] command line args: [] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 18 11:26:12.379557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 18 11:26:12.379642 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 18 11:26:12.379749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] command line args: [] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 18 11:26:12.379830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ================================================================================ {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 18 11:26:12.380023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 18 11:26:12.380123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] admin_token = **** {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.380196 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ================================================================================ {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 18 11:26:12.380274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] config_dir = [] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.380445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.380525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] admin_token = **** {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.380628 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] config_source = [] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.380714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] config_dir = [] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.380835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] debug = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.380951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.381015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.381327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] config_source = [] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.381327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] debug = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.381450 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.381520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.381609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.381675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.381821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.381925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.382029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.382126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.382265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] list_limit = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.382371 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.382472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] log_config_append = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.382567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] list_limit = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.382673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.382782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] log_config_append = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.382877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] log_dir = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.382970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.383064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] log_file = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.383170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] log_dir = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.383291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.383412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] log_file = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.383412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.383599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.383680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.383752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.383864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.383949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.384072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.384157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.384237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.384364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.384441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.384558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.384662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.384749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.384917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.385013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.385154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.385239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.385386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.385470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.385619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.385752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.385839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.385911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.386092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.386173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.386347 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.386436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.386626 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.386844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.387026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] publish_errors = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.387112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.387259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.387344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] publish_errors = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.387490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.387571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.387668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.387789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.387897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.387984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.388138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.388224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.388371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] standard_threads = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.388451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.388603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.388689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] standard_threads = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.388835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.388939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] use_journal = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] use_json = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] use_journal = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] use_stderr = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] use_json = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] use_syslog = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] use_stderr = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.389952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] use_syslog = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.390019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.390140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 18 11:26:12.390218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.390400 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.390488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.390640 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.390755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.391459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.391539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.391702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.391788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.391938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.392015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.392184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.392261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.392418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.392510 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.392662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.392739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.392931 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.393012 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.393184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.393274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.393417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.393494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.393661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.393744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.393894 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.393984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.394137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.394215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.394380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.394457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.394615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.394694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.394851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.394929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.395107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.395191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.395349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.395424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.395572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.395652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.395824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.395901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.396079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.396164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.396327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.396404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.396575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.396659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.396804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.396928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.397059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.397137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.397288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.397373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.397519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.397604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.397757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.397837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.398000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.398078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.398229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.398308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.398457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.398541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.398688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.398764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.398935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.399012 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.399169 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.399247 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.399406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.399483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.399634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.399713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.399871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.399948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.400108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.400195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.400335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.400414 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.400582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.400674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.400794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.400932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.401091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.401166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.401272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.401351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.401509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.401588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.401737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.401816 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.401974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.402052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.402204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.402281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.402433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.402544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.402649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.402745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.402896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.403013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.403130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.403233 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.403365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.403469 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.403600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.403686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.403836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.403923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.404068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.404181 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.404304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.404391 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.404534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.404612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.404761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.404861 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.405026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.405116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.405263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.405385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.405493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.405583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.405726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.405814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.405974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.406064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.406205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.406290 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.406466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.406544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.406699 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.406785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.406934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.407014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.407192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.407276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.407433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.407519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.407674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.407758 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.407911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.408026 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.408145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.408277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.408380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.408518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.408662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.408805 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.408988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.409119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.409247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.409370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.409498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.409625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.409752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.409885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.410014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.410096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.410256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.410383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.410508 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.410656 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.410783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.410910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.411039 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.411164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.411297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.411431 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.411559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.411686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.411812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.411938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.412067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.412201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.412330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.412466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.412592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.412716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.412856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.413059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.413189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.413316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.413441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.413573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.413710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.413843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.413977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.414104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.414232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.414373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.414501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.414640 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.414776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.414908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.415035 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.415161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.415287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.415413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.415551 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.415678 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.415804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.415928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.416065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.416190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.416328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.416454 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.416580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.416703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.416831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.416971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.417129 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] auth.external = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.417129 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.417389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.417389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] auth.external = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.417631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.417759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.417895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.417895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] auth.password = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.418143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.418272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] auth.token = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.418403 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.418527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.418662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] auth.password = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.418795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.418931 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] auth.token = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.419058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.419185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.419317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.419445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.419586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.419724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.419851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.419977 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.420103 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.420185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.420350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] credential.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.420487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.420623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.420749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.420894 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.421019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] credential.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.421147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.421285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.421428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.421557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.421685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.421813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.421941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.422075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.422205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.422331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.422459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.422588 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.422716 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.422852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.422979 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.423115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.423242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.423375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.423510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.423638 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.423772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.423906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.424034 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.424160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.424295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.424422 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.424549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] federation.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.424674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.424807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.424976 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.425116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.425244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] federation.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.425371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.425498 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.425626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.425753 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.425889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.426018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.426144 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.426273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.426401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.426528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.426673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.426802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.426936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.427066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.427194 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.427323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.427460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.427586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.427714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.427843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.427969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.428096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.428307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.428438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.428580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.428714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.428857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.428988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.429162 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.429299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.429426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.429554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.429681 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.429808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.429949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.430079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.430203 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.430330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.430457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.430582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.430714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.430839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.430964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.431088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.431212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.431337 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.431469 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.431594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.431717 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.431847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.431972 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.432107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.432239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.432365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.432490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.432613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.432738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.432909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.433047 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.433177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.433304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.433432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.433560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.433694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.433832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.433959 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.434094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.434221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.434348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.434483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.434610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.434737 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.434863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.434990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.435119 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.435254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.435381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.435508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.435645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.435772 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.435897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.436030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.436158 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.436284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.436409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.436537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.436662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.436662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.436929 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.437054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.437184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.437313 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.437450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.438092 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.438170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.438237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.438307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.438368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.438546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.438683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.438816 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.438946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.439078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.439206 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.439336 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.439473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.439610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.439737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.439864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.439990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.440115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.440253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.440338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.440503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.440627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.440751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.440944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.441083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.441211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.441348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.441475 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.441604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.441732 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.441866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.441996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.442123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.442249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.442376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.442502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.442637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.442765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.442893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.443018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.443153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.443281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.443416 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.443545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.443673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.443800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.443926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.444052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.444186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.444333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.444457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.444586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.444712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.444868 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.445006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.445137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.445266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.445394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.445522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.445652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.445789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.445937 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.446061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.446189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.446318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.446446 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.446583 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.446721 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.446851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.446979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.447106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.447234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.447383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.447507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.447634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.447760 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.447890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.448014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.448151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.448284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.448414 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.448552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.448677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.448803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.448973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.449095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.449224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.449358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.449488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.449616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.449752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.449883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.450012 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.450140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.450275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.450409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.450536 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.450665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.450794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.450923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.451052 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] resource.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.451187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.451315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.451444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.451572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.451719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.451844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.451981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] resource.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.452122 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.452249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.452377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.452506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.452630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.452767 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.452918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.453043 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.453202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.453841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.453972 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.454168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.454295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] role.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.454420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.454555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] role.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.454686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.454950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.455082 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.455206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] role.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.455335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.455461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] role.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.455585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.455722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.455853 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.455989 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.456115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.456251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.456376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.456509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.456639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.456769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.456942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.457073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.457203 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.457359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.457485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.457610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.457740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.457869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.457997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.458142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.458276 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.458404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.458530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.458658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.458785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.458918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.459048 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.459174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.459303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.459303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.459550 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.459550 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.459797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.459797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.460039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.460166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.460300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.460428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.460556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.460634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.460706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.460777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.460877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.460956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461329 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461550 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461772 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461847 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461919 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.461991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462062 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] token.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462505 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462577 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] token.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.462947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463616 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.463984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.connection = **** {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.464926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.connection = **** {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465915 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.465987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466655 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.466950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 18 11:26:12.467677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 18 11:26:12.467677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 18 11:26:12.495190 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.495419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.495645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.496281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.496518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.496748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.504615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.505695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.533135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.533362 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.534233 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.534467 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.587475 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.588267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.636824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.637077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.637959 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.638197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.797705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.797875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.798066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.798284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.798422 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.798510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.798657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.798739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.798883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.799105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.799323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.799537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.822962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.823141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.823276 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.823405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.823488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.823562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.823634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.823706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.823787 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.823916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.823999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.824072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.824195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.824285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.836513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.836654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.850399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.850600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.850705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.850838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.850918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.851065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.851143 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.851288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.851376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.851502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.851582 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.851714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.851793 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.851931 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 18 11:26:12.868990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.869199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 18 11:26:12.869330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.869544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 18 11:26:12.869660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.869741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 18 11:26:12.869887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.870005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.870087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 18 11:26:12.870274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.870377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.870536 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.870632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.870800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.870895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.871076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.880518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.880997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.881277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.881492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.881625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.881780 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.882002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.882286 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.888592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.889890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.892213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.892502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.892790 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.893106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.893378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.893473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.893652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.893732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.893927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.894007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.894205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.894300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.894480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.894560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.894740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.894830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.895011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.895091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.895284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.895555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.895831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.896101 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.918416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.919364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.919888 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.920180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.920836 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.921145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.924983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.925314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.925559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.925884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.926171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.926452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.932007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.932301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.932799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.933112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.938595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.938887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.939173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.939358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.939440 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.939658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.939740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.939948 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.940030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.940239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.940329 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.940508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.940589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.940779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.941072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.941343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.956761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.957077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.957571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.957870 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.961073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.961856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.964386 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.964693 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.965193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.965499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.969037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.969835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.972180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.973019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.975520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.975828 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.976117 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.976267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.976406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.976571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.976689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.976878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.977019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.977171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.977469 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.977758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.993316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.993619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.993889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.993969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:12.994153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.994252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.994506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:12.994773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.001607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.002201 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.003159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.003744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.004705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.005006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.005295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.005572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.007713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.008009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.008230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.008311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.008532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.008627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.008803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.008962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.009108 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.009508 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.020835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.021146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.021228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.021434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.021528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.021720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.021802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.022004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.022108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.022386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.022479 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.022668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.022750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.022949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.038199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.038349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.041159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.041287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.044069 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.044233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.045544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.045718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.047045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.047210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.047321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.047495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.047772 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.047898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.054124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.054292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.055597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.055760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.058541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.058753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.058942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.059151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.059244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.059442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.059522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.059723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.067869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.068073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.068169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.068369 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.068450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.068654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.075333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.075576 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 18 11:26:13.075661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.075865 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.075946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.076152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.076276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.076425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.076518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.076695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.076774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.076991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.077086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.077279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.077368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.077552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.077640 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.077820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.077900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.078096 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.078758 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.078919 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.079002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.079198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.079333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.079481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.079624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.079896 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 18 11:26:13.115088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x562b3f0039c0 pid: 81701 (default app) Oct 18 11:26:13.115202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x562b3f0039c0 pid: 81700 (default app) Oct 18 11:26:13.118460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16a81e-a196-4c9e-bd1e-9913602de42a None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:13.118747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16a81e-a196-4c9e-bd1e-9913602de42a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:13.119018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16a81e-a196-4c9e-bd1e-9913602de42a None None] PATH_INFO: `/v3` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:13.123186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Tue Oct 18 11:26:13 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:19.910410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca7444-6deb-40f8-b8ba-a83a361e19a6 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:19.910948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca7444-6deb-40f8-b8ba-a83a361e19a6 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:19.910948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca7444-6deb-40f8-b8ba-a83a361e19a6 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:19.915228 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:26:19.920495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:19.921034 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:19.921131 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:19.992524 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: INFO dbcounter [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] Registered counter for database keystone Oct 18 11:26:20.011623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG oslo_db.sqlalchemy.engines [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81701) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 18 11:26:20.017985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writer thread running {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 18 11:26:20.197399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG passlib.handlers.bcrypt [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81701) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 18 11:26:20.197791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG passlib.handlers.bcrypt [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81701) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 18 11:26:20.256558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:20.258208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:20.385101 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 18 11:26:20.856283 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca7444-6deb-40f8-b8ba-a83a361e19a6 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:20.926652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: INFO dbcounter [None req-56ca7444-6deb-40f8-b8ba-a83a361e19a6 None None] Registered counter for database keystone Oct 18 11:26:20.948173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG oslo_db.sqlalchemy.engines [None req-56ca7444-6deb-40f8-b8ba-a83a361e19a6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81700) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 18 11:26:20.955058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writer thread running {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 18 11:26:21.081157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca7444-6deb-40f8-b8ba-a83a361e19a6 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:21.085615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090f26d-2aa2-4d46-9dd3-f36b1cc36a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:21.086252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090f26d-2aa2-4d46-9dd3-f36b1cc36a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:21.086533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090f26d-2aa2-4d46-9dd3-f36b1cc36a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:21.086807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090f26d-2aa2-4d46-9dd3-f36b1cc36a40 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:21.095855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090f26d-2aa2-4d46-9dd3-f36b1cc36a40 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:21.148248 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e090f26d-2aa2-4d46-9dd3-f36b1cc36a40 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 18 11:26:21.148597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090f26d-2aa2-4d46-9dd3-f36b1cc36a40 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:21.159023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Tue Oct 18 11:26:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 18 11:26:21.165043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:21.182379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e81e0-c14c-41b3-8891-455a707adf67 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:21.186912 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646667f-e809-4fd9-bd05-f272b2d6aa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:21.188569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646667f-e809-4fd9-bd05-f272b2d6aa79 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:21.188876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646667f-e809-4fd9-bd05-f272b2d6aa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:21.189146 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646667f-e809-4fd9-bd05-f272b2d6aa79 admin admin] PATH_INFO: `/v3/projects/4c023e59351c4a06998c34820bba9111` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:21.190756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646667f-e809-4fd9-bd05-f272b2d6aa79 admin admin] RBAC: Authorizing `identity:get_project(project_id=4c023e59351c4a06998c34820bba9111)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:21.244697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d646667f-e809-4fd9-bd05-f272b2d6aa79 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 18 11:26:21.245076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646667f-e809-4fd9-bd05-f272b2d6aa79 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:21.249799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Tue Oct 18 11:26:21 2022] GET /identity/v3/projects/4c023e59351c4a06998c34820bba9111 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:23.390072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bab564-b764-439a-9a5f-3dde7850fd6e None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:23.391122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bab564-b764-439a-9a5f-3dde7850fd6e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:23.391226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bab564-b764-439a-9a5f-3dde7850fd6e None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:23.395273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:26:23.401217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6e662-aea5-40e4-bde1-f56fbd744ead None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:23.401694 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6e662-aea5-40e4-bde1-f56fbd744ead None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:23.401789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6e662-aea5-40e4-bde1-f56fbd744ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:23.422028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b6b6e662-aea5-40e4-bde1-f56fbd744ead None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:23.423670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b6b6e662-aea5-40e4-bde1-f56fbd744ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:23.431467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 18 11:26:23.697874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bab564-b764-439a-9a5f-3dde7850fd6e None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:23.710438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bab564-b764-439a-9a5f-3dde7850fd6e None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:23.714705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce89ae7-cf6c-40f8-88e4-305e25ca98c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:23.715307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89ae7-cf6c-40f8-88e4-305e25ca98c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:23.715587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89ae7-cf6c-40f8-88e4-305e25ca98c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:23.715860 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89ae7-cf6c-40f8-88e4-305e25ca98c1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:23.723401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce89ae7-cf6c-40f8-88e4-305e25ca98c1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:23.725208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce89ae7-cf6c-40f8-88e4-305e25ca98c1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:23.733793 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Tue Oct 18 11:26:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:26:23.739210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6e662-aea5-40e4-bde1-f56fbd744ead None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:23.748337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6e662-aea5-40e4-bde1-f56fbd744ead None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:23.753171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef3638-e46a-459e-903e-825f031dcca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:23.753794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef3638-e46a-459e-903e-825f031dcca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:23.754072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef3638-e46a-459e-903e-825f031dcca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:23.754339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef3638-e46a-459e-903e-825f031dcca8 admin admin] PATH_INFO: `/v3/users/c5f193bd980d483b8200f674f5249745` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:23.755624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef3638-e46a-459e-903e-825f031dcca8 admin admin] RBAC: Authorizing `identity:get_user(user_id=c5f193bd980d483b8200f674f5249745)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:23.757494 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef3638-e46a-459e-903e-825f031dcca8 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:23.758999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Tue Oct 18 11:26:23 2022] GET /identity/v3/users/c5f193bd980d483b8200f674f5249745 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:25.971059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54642446-b361-4f9e-86b7-421e1a3224a4 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:25.971806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54642446-b361-4f9e-86b7-421e1a3224a4 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:25.971806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54642446-b361-4f9e-86b7-421e1a3224a4 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:25.972947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:25.979098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6577e-ab84-4996-ab7c-41d858057d31 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:25.979701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6577e-ab84-4996-ab7c-41d858057d31 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:25.980040 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6577e-ab84-4996-ab7c-41d858057d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:25.998822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9ed6577e-ab84-4996-ab7c-41d858057d31 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:26.000557 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9ed6577e-ab84-4996-ab7c-41d858057d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:26.017843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f004a2-9df7-4aa8-bf4c-55f270eaacd8 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:26.018122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f004a2-9df7-4aa8-bf4c-55f270eaacd8 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:26.018394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f004a2-9df7-4aa8-bf4c-55f270eaacd8 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:26.019715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:26:26.025142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358af1d7-6932-44c4-92c0-de4d091a919f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:26.025437 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358af1d7-6932-44c4-92c0-de4d091a919f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:26.025710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358af1d7-6932-44c4-92c0-de4d091a919f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:26.068908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG passlib.handlers.bcrypt [None req-358af1d7-6932-44c4-92c0-de4d091a919f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81700) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 18 11:26:26.069296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG passlib.handlers.bcrypt [None req-358af1d7-6932-44c4-92c0-de4d091a919f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81700) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 18 11:26:26.091480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:26.118551 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-358af1d7-6932-44c4-92c0-de4d091a919f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:26.120149 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-358af1d7-6932-44c4-92c0-de4d091a919f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:26.227059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:26:26.359667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6577e-ab84-4996-ab7c-41d858057d31 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:26.385862 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:26.474601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6577e-ab84-4996-ab7c-41d858057d31 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:26.492756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358af1d7-6932-44c4-92c0-de4d091a919f None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:26.511413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36744b1-c060-4e7b-990f-fa132e208711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:26.512069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36744b1-c060-4e7b-990f-fa132e208711 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:26.512352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36744b1-c060-4e7b-990f-fa132e208711 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:26.512637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36744b1-c060-4e7b-990f-fa132e208711 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:26.525640 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36744b1-c060-4e7b-990f-fa132e208711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-b36744b1-c060-4e7b-990f-fa132e208711 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:26.534648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:26.535923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:26:26.536756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:26:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:26.541665 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36744b1-c060-4e7b-990f-fa132e208711 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:26.600132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358af1d7-6932-44c4-92c0-de4d091a919f None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:26.602538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:26.634070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:26.640084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bbadc-8b78-4d76-969a-2b1dafe38e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:26.641857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bbadc-8b78-4d76-969a-2b1dafe38e51 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:26.642530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bbadc-8b78-4d76-969a-2b1dafe38e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:26.642816 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bbadc-8b78-4d76-969a-2b1dafe38e51 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:26.645166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bbadc-8b78-4d76-969a-2b1dafe38e51 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:26.649021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bbadc-8b78-4d76-969a-2b1dafe38e51 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:26.655193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36744b1-c060-4e7b-990f-fa132e208711 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:26.663310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.sql.core [None req-146bbadc-8b78-4d76-969a-2b1dafe38e51 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 18 11:26:26.663310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 18 11:26:26.663310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [parameters: {'id': 'a147ce4791ae4391bb4fe4a9bd79ec83', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 18 11:26:26.663310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81700) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-146bbadc-8b78-4d76-969a-2b1dafe38e51 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application result = self._query(query) Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application conn.query(q) Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 18 11:26:26.681776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application result.read() Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 18 11:26:26.682809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application next(self.gen) Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application yield resource Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application next(self.gen) Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self.session.rollback() Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self.force_reraise() Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise self.value Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application session.commit() Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 18 11:26:26.683946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._prepare_impl() Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self.session.flush() Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._flush(objects) Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application compat.raise_( Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application flush_context.execute() Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rec.execute(self) Oct 18 11:26:26.685134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 18 11:26:26.686180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application result = self._query(query) Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application conn.query(q) Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application result.read() Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 18 11:26:26.687165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [parameters: {'id': 'a147ce4791ae4391bb4fe4a9bd79ec83', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 18 11:26:26.688819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:26.689679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 8/15] 10.222.0.17 () {60 vars in 1137 bytes} [Tue Oct 18 11:26:26 2022] POST /identity/v3/domains => generated 190 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:26:26.690628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bbadc-8b78-4d76-969a-2b1dafe38e51 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:26.705538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663079c-15c2-4119-94bd-581100904501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:26.706151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663079c-15c2-4119-94bd-581100904501 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:26.706423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663079c-15c2-4119-94bd-581100904501 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:26.706690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663079c-15c2-4119-94bd-581100904501 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:26.707363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663079c-15c2-4119-94bd-581100904501 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:26.707660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663079c-15c2-4119-94bd-581100904501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:26.709061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663079c-15c2-4119-94bd-581100904501 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:26.721098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 8/16] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:26:26.726779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663079c-15c2-4119-94bd-581100904501 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:26.792115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bbadc-8b78-4d76-969a-2b1dafe38e51 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:26.830168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474940b2-81a3-4944-bf1a-bbf61f512a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:26.831035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474940b2-81a3-4944-bf1a-bbf61f512a21 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:26.831309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474940b2-81a3-4944-bf1a-bbf61f512a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:26.831574 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474940b2-81a3-4944-bf1a-bbf61f512a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:26.834347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663079c-15c2-4119-94bd-581100904501 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:26.849949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474940b2-81a3-4944-bf1a-bbf61f512a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:26.851467 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474940b2-81a3-4944-bf1a-bbf61f512a21 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-474940b2-81a3-4944-bf1a-bbf61f512a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:26:26.869373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:26.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:26:26.871080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:26:26.871080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:26:26.871080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:26.871080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:26:26.871719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bf1ac-a31c-412f-b5ca-a1f3364e4a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:26.872336 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bf1ac-a31c-412f-b5ca-a1f3364e4a2d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:26.872623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bf1ac-a31c-412f-b5ca-a1f3364e4a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:26.873586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bf1ac-a31c-412f-b5ca-a1f3364e4a2d None admin] PATH_INFO: `/v3/users/c5f193bd980d483b8200f674f5249745` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:26.874100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bf1ac-a31c-412f-b5ca-a1f3364e4a2d None admin] RBAC: Authorizing `identity:get_user(user_id=c5f193bd980d483b8200f674f5249745)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:26.876338 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bf1ac-a31c-412f-b5ca-a1f3364e4a2d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:26.876922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474940b2-81a3-4944-bf1a-bbf61f512a21 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:26.877979 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:26 2022] GET /identity/v3/users/c5f193bd980d483b8200f674f5249745 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:26.883286 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bf1ac-a31c-412f-b5ca-a1f3364e4a2d None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:26.898504 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:26.902154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:26.994686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474940b2-81a3-4944-bf1a-bbf61f512a21 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:26.995352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bf1ac-a31c-412f-b5ca-a1f3364e4a2d None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:27.035161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff68012-8e25-40a4-ba1f-9758c08b915c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:27.036131 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff68012-8e25-40a4-ba1f-9758c08b915c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:27.036415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff68012-8e25-40a4-ba1f-9758c08b915c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:27.036578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3176b6b-c8d3-43b5-a620-72b5139ebc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:27.036876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff68012-8e25-40a4-ba1f-9758c08b915c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:27.037219 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3176b6b-c8d3-43b5-a620-72b5139ebc9b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:27.037493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3176b6b-c8d3-43b5-a620-72b5139ebc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:27.037761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3176b6b-c8d3-43b5-a620-72b5139ebc9b None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:27.038174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff68012-8e25-40a4-ba1f-9758c08b915c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:27.038406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3176b6b-c8d3-43b5-a620-72b5139ebc9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:27.038703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3176b6b-c8d3-43b5-a620-72b5139ebc9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:27.039633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff68012-8e25-40a4-ba1f-9758c08b915c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:27.040068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3176b6b-c8d3-43b5-a620-72b5139ebc9b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:27.041262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:26:27.048318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff68012-8e25-40a4-ba1f-9758c08b915c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:27.059683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:26:27.146413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:27.170405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff68012-8e25-40a4-ba1f-9758c08b915c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:27.214770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087995ff-1362-4121-8546-0a20e138a3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:27.215466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087995ff-1362-4121-8546-0a20e138a3fe None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:27.215744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087995ff-1362-4121-8546-0a20e138a3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:27.216020 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087995ff-1362-4121-8546-0a20e138a3fe None admin] PATH_INFO: `/v3/domains/default/users/c5f193bd980d483b8200f674f5249745/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:27.218127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087995ff-1362-4121-8546-0a20e138a3fe None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c5f193bd980d483b8200f674f5249745, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:27.219652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087995ff-1362-4121-8546-0a20e138a3fe None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:27.338226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 11/21] 10.222.0.17 () {58 vars in 1361 bytes} [Tue Oct 18 11:26:27 2022] PUT /identity/v3/domains/default/users/c5f193bd980d483b8200f674f5249745/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:26:29.892831 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1f169-bd94-46c6-bb89-cad12e688367 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:29.893708 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1f169-bd94-46c6-bb89-cad12e688367 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:29.894128 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1f169-bd94-46c6-bb89-cad12e688367 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:29.895909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:29.911741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93238d17-1d58-4603-aef2-2dd60ebf4827 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:29.912225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93238d17-1d58-4603-aef2-2dd60ebf4827 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:29.912296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93238d17-1d58-4603-aef2-2dd60ebf4827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:29.918220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3fb85-ddde-4017-bc22-76ffdb818847 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:29.918780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3fb85-ddde-4017-bc22-76ffdb818847 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:29.919248 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3fb85-ddde-4017-bc22-76ffdb818847 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:29.920782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:26:29.927101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b8c4-86ea-4885-a8f3-ea037d511508 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:29.927661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b8c4-86ea-4885-a8f3-ea037d511508 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:29.928157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b8c4-86ea-4885-a8f3-ea037d511508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:29.938109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-93238d17-1d58-4603-aef2-2dd60ebf4827 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:29.939705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-93238d17-1d58-4603-aef2-2dd60ebf4827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:29.962074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-63b1b8c4-86ea-4885-a8f3-ea037d511508 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:29.963674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-63b1b8c4-86ea-4885-a8f3-ea037d511508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:30.095296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:30.101197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 13/25] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:30.309064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8fcd1-5168-4812-a7e9-4236dc38956b None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:30.309764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8fcd1-5168-4812-a7e9-4236dc38956b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:30.310278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8fcd1-5168-4812-a7e9-4236dc38956b None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:30.315149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 13/26] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:30.320616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef86689-6ae2-4245-9396-677dfd7b953d None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:30.320944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef86689-6ae2-4245-9396-677dfd7b953d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:30.321223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef86689-6ae2-4245-9396-677dfd7b953d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:30.345880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-8ef86689-6ae2-4245-9396-677dfd7b953d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:30.347134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8ef86689-6ae2-4245-9396-677dfd7b953d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:30.354215 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:30.433656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32092f-e6c2-4943-9302-c98be175c0f6 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:30.434366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32092f-e6c2-4943-9302-c98be175c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:30.434865 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32092f-e6c2-4943-9302-c98be175c0f6 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:30.441056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 14/27] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:30.445438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 14/28] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:30.447495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd235b2-59bf-4142-8a68-9ba33c00545a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:30.448019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd235b2-59bf-4142-8a68-9ba33c00545a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:30.448512 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd235b2-59bf-4142-8a68-9ba33c00545a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:30.485866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7cd235b2-59bf-4142-8a68-9ba33c00545a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:30.487439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7cd235b2-59bf-4142-8a68-9ba33c00545a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:30.575229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:30.580787 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 15/29] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:30.633851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef86689-6ae2-4245-9396-677dfd7b953d None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:30.747417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef86689-6ae2-4245-9396-677dfd7b953d None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:30.786206 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd235b2-59bf-4142-8a68-9ba33c00545a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:30.788716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800c7cf-3a1e-4fcf-bc6b-c89eae946285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:30.789456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800c7cf-3a1e-4fcf-bc6b-c89eae946285 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:30.789741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800c7cf-3a1e-4fcf-bc6b-c89eae946285 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:30.790018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800c7cf-3a1e-4fcf-bc6b-c89eae946285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:30.800754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:30.803086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800c7cf-3a1e-4fcf-bc6b-c89eae946285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:30.807068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800c7cf-3a1e-4fcf-bc6b-c89eae946285 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-2800c7cf-3a1e-4fcf-bc6b-c89eae946285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:26:30.822508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:30.823410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:26:30.824204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:26:30.824204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:26:30.824204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:30.825709 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 15/30] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:30.826269 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800c7cf-3a1e-4fcf-bc6b-c89eae946285 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:30.957687 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd235b2-59bf-4142-8a68-9ba33c00545a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:30.975272 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800c7cf-3a1e-4fcf-bc6b-c89eae946285 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:31.004811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461970dc-1950-4150-b568-3a1f089b976a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:31.005443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461970dc-1950-4150-b568-3a1f089b976a None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.005728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461970dc-1950-4150-b568-3a1f089b976a None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.006006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461970dc-1950-4150-b568-3a1f089b976a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.017954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd94abb-915f-4f1a-b07c-12f1c3f5f298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:31.018152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461970dc-1950-4150-b568-3a1f089b976a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:31.018893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd94abb-915f-4f1a-b07c-12f1c3f5f298 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.019317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd94abb-915f-4f1a-b07c-12f1c3f5f298 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.019727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd94abb-915f-4f1a-b07c-12f1c3f5f298 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.020575 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd94abb-915f-4f1a-b07c-12f1c3f5f298 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:31.022165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd94abb-915f-4f1a-b07c-12f1c3f5f298 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-461970dc-1950-4150-b568-3a1f089b976a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:31.022934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:31.023795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 16/31] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:26:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:31.029406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461970dc-1950-4150-b568-3a1f089b976a None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:31.092528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:31.138882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461970dc-1950-4150-b568-3a1f089b976a None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:31.157915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 16/32] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Oct 18 11:26:30 2022] POST /identity/v3/roles => generated 219 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:31.159197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32350f41-4c58-4995-8fd7-c5ac694f0299 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.159493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32350f41-4c58-4995-8fd7-c5ac694f0299 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.159775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32350f41-4c58-4995-8fd7-c5ac694f0299 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.165349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 17/33] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:31.165349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32350f41-4c58-4995-8fd7-c5ac694f0299 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:31.188669 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111df434-8219-41d8-ae3d-db85af5beb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:31.189738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111df434-8219-41d8-ae3d-db85af5beb2e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.190207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111df434-8219-41d8-ae3d-db85af5beb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.190718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111df434-8219-41d8-ae3d-db85af5beb2e None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.191649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111df434-8219-41d8-ae3d-db85af5beb2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:31.192194 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111df434-8219-41d8-ae3d-db85af5beb2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:31.194099 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111df434-8219-41d8-ae3d-db85af5beb2e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:31.215179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 17/34] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:31.216522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111df434-8219-41d8-ae3d-db85af5beb2e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:31.241552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:31.300212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32350f41-4c58-4995-8fd7-c5ac694f0299 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:31.341125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111df434-8219-41d8-ae3d-db85af5beb2e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:31.342419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c1b4f-be37-47fd-a0f7-08b089c0b488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:31.343075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c1b4f-be37-47fd-a0f7-08b089c0b488 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.343362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c1b4f-be37-47fd-a0f7-08b089c0b488 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.344205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c1b4f-be37-47fd-a0f7-08b089c0b488 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.344936 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c1b4f-be37-47fd-a0f7-08b089c0b488 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:31.346499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c1b4f-be37-47fd-a0f7-08b089c0b488 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:31.365053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:31.368408 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 18/35] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Oct 18 11:26:31 2022] POST /identity/v3/roles => generated 225 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:31.370078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6c959-91d4-4092-abcc-f7faebb98131 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.370645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6c959-91d4-4092-abcc-f7faebb98131 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.371138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6c959-91d4-4092-abcc-f7faebb98131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.396160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c2b6c959-91d4-4092-abcc-f7faebb98131 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:31.398427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c2b6c959-91d4-4092-abcc-f7faebb98131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:31.405032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3eb90-7eaf-42be-a0aa-1f72edd4e33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:31.406114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3eb90-7eaf-42be-a0aa-1f72edd4e33f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.407095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3eb90-7eaf-42be-a0aa-1f72edd4e33f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.407567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3eb90-7eaf-42be-a0aa-1f72edd4e33f None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.408470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3eb90-7eaf-42be-a0aa-1f72edd4e33f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:31.409668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3eb90-7eaf-42be-a0aa-1f72edd4e33f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:31.411986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3eb90-7eaf-42be-a0aa-1f72edd4e33f None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:31.424312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 18/36] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:31.426059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3eb90-7eaf-42be-a0aa-1f72edd4e33f None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:31.493186 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 19/37] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:31.495021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6c959-91d4-4092-abcc-f7faebb98131 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:31.561527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:31.570132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3eb90-7eaf-42be-a0aa-1f72edd4e33f None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:31.620111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d683c-7e76-4b2f-a810-0f05cef34a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:31.621056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d683c-7e76-4b2f-a810-0f05cef34a91 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.621487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d683c-7e76-4b2f-a810-0f05cef34a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.621943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d683c-7e76-4b2f-a810-0f05cef34a91 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.624045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d683c-7e76-4b2f-a810-0f05cef34a91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:31.625960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d683c-7e76-4b2f-a810-0f05cef34a91 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:31.644971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6c959-91d4-4092-abcc-f7faebb98131 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:31.668353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 19/38] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:26:31 2022] POST /identity/v3/projects => generated 301 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:31.670314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd4302-0fe5-4a1f-b071-6efb4132270b None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.671073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd4302-0fe5-4a1f-b071-6efb4132270b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.671557 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd4302-0fe5-4a1f-b071-6efb4132270b None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.673869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 20/39] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:31.686537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f144f6-9440-48ff-8886-ffd70280e267 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.687918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f144f6-9440-48ff-8886-ffd70280e267 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.688375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f144f6-9440-48ff-8886-ffd70280e267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.704448 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349659b8-9867-4f6e-9473-4b52adc8cdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:31.706223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349659b8-9867-4f6e-9473-4b52adc8cdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.706835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349659b8-9867-4f6e-9473-4b52adc8cdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.707300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349659b8-9867-4f6e-9473-4b52adc8cdf1 None admin] PATH_INFO: `/v3/users/c5f193bd980d483b8200f674f5249745` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.709380 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349659b8-9867-4f6e-9473-4b52adc8cdf1 None admin] RBAC: Authorizing `identity:get_user(user_id=c5f193bd980d483b8200f674f5249745)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:31.711196 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349659b8-9867-4f6e-9473-4b52adc8cdf1 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:31.713418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 20/40] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:31 2022] GET /identity/v3/users/c5f193bd980d483b8200f674f5249745 => generated 247 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:31.719574 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349659b8-9867-4f6e-9473-4b52adc8cdf1 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:31.829010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b1f144f6-9440-48ff-8886-ffd70280e267 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:31.830158 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b1f144f6-9440-48ff-8886-ffd70280e267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:31.876930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:31.893648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349659b8-9867-4f6e-9473-4b52adc8cdf1 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:31.913645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:31.934625 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 21/41] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:31.935673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f144f6-9440-48ff-8886-ffd70280e267 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:31.959574 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671d2c3-d40c-4558-95f6-39e97ff5bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:31.960689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d2c3-d40c-4558-95f6-39e97ff5bd77 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:31.961179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d2c3-d40c-4558-95f6-39e97ff5bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:31.961625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d2c3-d40c-4558-95f6-39e97ff5bd77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:31.964150 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671d2c3-d40c-4558-95f6-39e97ff5bd77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:31.967587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671d2c3-d40c-4558-95f6-39e97ff5bd77 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:31.972719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 21/42] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:26:31.976980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671d2c3-d40c-4558-95f6-39e97ff5bd77 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:32.076588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f144f6-9440-48ff-8886-ffd70280e267 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:32.102774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671d2c3-d40c-4558-95f6-39e97ff5bd77 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:32.114358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f82dc4-8d69-4193-9288-387597d24207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:32.115250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f82dc4-8d69-4193-9288-387597d24207 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:32.115534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f82dc4-8d69-4193-9288-387597d24207 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:32.115810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f82dc4-8d69-4193-9288-387597d24207 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:32.116496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f82dc4-8d69-4193-9288-387597d24207 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:32.117945 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f82dc4-8d69-4193-9288-387597d24207 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:32.147334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 22/43] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Oct 18 11:26:31 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:32.150476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fda14d-7448-4266-90d5-96ff9e4eaba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:32.151134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fda14d-7448-4266-90d5-96ff9e4eaba6 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:32.151413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fda14d-7448-4266-90d5-96ff9e4eaba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:32.151694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fda14d-7448-4266-90d5-96ff9e4eaba6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:32.152462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fda14d-7448-4266-90d5-96ff9e4eaba6 None admin] RBAC: Adding query filter params (role.id=8d9ff9da6ad946a6bc49336777182ff9, scope.domain.id=default, user.id=c5f193bd980d483b8200f674f5249745) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:32.152812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fda14d-7448-4266-90d5-96ff9e4eaba6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:32.154337 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fda14d-7448-4266-90d5-96ff9e4eaba6 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:32.167999 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 22/44] 10.222.0.17 () {56 vars in 1322 bytes} [Tue Oct 18 11:26:31 2022] GET /identity/v3/role_assignments?user.id=c5f193bd980d483b8200f674f5249745&scope.domain.id=default&role.id=8d9ff9da6ad946a6bc49336777182ff9 => generated 530 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:26:32.208219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f82dc4-8d69-4193-9288-387597d24207 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:32.240411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:32.339830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f82dc4-8d69-4193-9288-387597d24207 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:32.381148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714a3bf-7672-4bd4-9a40-568fdb25536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:32.381801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714a3bf-7672-4bd4-9a40-568fdb25536f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:32.382083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714a3bf-7672-4bd4-9a40-568fdb25536f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:32.382354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714a3bf-7672-4bd4-9a40-568fdb25536f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:32.383500 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714a3bf-7672-4bd4-9a40-568fdb25536f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:32.385003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714a3bf-7672-4bd4-9a40-568fdb25536f None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:32.386623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 23/45] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:32.392282 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fda14d-7448-4266-90d5-96ff9e4eaba6 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:32.439612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:32.493931 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fda14d-7448-4266-90d5-96ff9e4eaba6 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:32.527133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d582d-6810-4fe9-82bf-1e24cc44ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:32.527734 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d582d-6810-4fe9-82bf-1e24cc44ecf9 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:32.528010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d582d-6810-4fe9-82bf-1e24cc44ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:32.528279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d582d-6810-4fe9-82bf-1e24cc44ecf9 None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:32.528952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d582d-6810-4fe9-82bf-1e24cc44ecf9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:32.530385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d582d-6810-4fe9-82bf-1e24cc44ecf9 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:32.556130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 23/46] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:26:32 2022] POST /identity/v3/projects => generated 312 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:34.681552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f311fe-95b6-46d7-8130-16ebff25b853 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:34.682060 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f311fe-95b6-46d7-8130-16ebff25b853 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:34.682141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f311fe-95b6-46d7-8130-16ebff25b853 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:34.683472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 24/47] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:34.689337 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4574f-4b6e-4500-8954-63151dd3d646 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:34.689752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4574f-4b6e-4500-8954-63151dd3d646 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:34.689882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4574f-4b6e-4500-8954-63151dd3d646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:34.707547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-02e4574f-4b6e-4500-8954-63151dd3d646 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:34.708722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-02e4574f-4b6e-4500-8954-63151dd3d646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:34.775800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 24/48] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:35.050445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f311fe-95b6-46d7-8130-16ebff25b853 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:35.142860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:35.152714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f311fe-95b6-46d7-8130-16ebff25b853 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:35.187715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f3c0a-8a94-4d68-a9ba-c6149bbb775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:35.188351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f3c0a-8a94-4d68-a9ba-c6149bbb775c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:35.188624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f3c0a-8a94-4d68-a9ba-c6149bbb775c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:35.188919 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f3c0a-8a94-4d68-a9ba-c6149bbb775c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:35.190021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f3c0a-8a94-4d68-a9ba-c6149bbb775c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:35.191474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f3c0a-8a94-4d68-a9ba-c6149bbb775c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:35.193054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 25/49] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:35.198617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4574f-4b6e-4500-8954-63151dd3d646 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:35.222703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:35.294183 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4574f-4b6e-4500-8954-63151dd3d646 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:35.325673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2097a0-a997-476b-ae12-2ce51dc360ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:35.326214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2097a0-a997-476b-ae12-2ce51dc360ac None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:35.326497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2097a0-a997-476b-ae12-2ce51dc360ac None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:35.326768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2097a0-a997-476b-ae12-2ce51dc360ac None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:35.327409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2097a0-a997-476b-ae12-2ce51dc360ac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:35.329425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2097a0-a997-476b-ae12-2ce51dc360ac None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:35.354300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 25/50] 10.222.0.17 () {60 vars in 1140 bytes} [Tue Oct 18 11:26:35 2022] POST /identity/v3/projects => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:37.496831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a4a9-fd6e-46e8-91ba-e0dc48c95a01 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:37.497366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a4a9-fd6e-46e8-91ba-e0dc48c95a01 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:37.497366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a4a9-fd6e-46e8-91ba-e0dc48c95a01 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:37.498697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 26/51] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:37.504400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1909e-1ec3-4d3c-b4d4-3d1c13a9b797 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:37.504792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1909e-1ec3-4d3c-b4d4-3d1c13a9b797 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:37.505007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1909e-1ec3-4d3c-b4d4-3d1c13a9b797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:37.523189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e0f1909e-1ec3-4d3c-b4d4-3d1c13a9b797 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:37.524372 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e0f1909e-1ec3-4d3c-b4d4-3d1c13a9b797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:37.589212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 26/52] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:37.858607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4a4a9-fd6e-46e8-91ba-e0dc48c95a01 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:37.957990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4a4a9-fd6e-46e8-91ba-e0dc48c95a01 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:37.991836 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ad5e8-a076-485f-857d-b9b08b687569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:37.992470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad5e8-a076-485f-857d-b9b08b687569 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:37.992750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad5e8-a076-485f-857d-b9b08b687569 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:37.993061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ad5e8-a076-485f-857d-b9b08b687569 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:37.994167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ad5e8-a076-485f-857d-b9b08b687569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:37.995619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ad5e8-a076-485f-857d-b9b08b687569 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:37.997222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 27/53] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:38.003360 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1909e-1ec3-4d3c-b4d4-3d1c13a9b797 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:38.011432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:38.101751 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1909e-1ec3-4d3c-b4d4-3d1c13a9b797 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:38.239348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6a923-4c3a-4c32-a4d7-125519fdfd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:38.240224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6a923-4c3a-4c32-a4d7-125519fdfd90 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:38.240509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6a923-4c3a-4c32-a4d7-125519fdfd90 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:38.240786 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6a923-4c3a-4c32-a4d7-125519fdfd90 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:38.241486 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6a923-4c3a-4c32-a4d7-125519fdfd90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:38.242938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6a923-4c3a-4c32-a4d7-125519fdfd90 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:38.268187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 27/54] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:26:38 2022] POST /identity/v3/users => generated 275 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:41.518910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ba586-84ab-4c9c-9921-cc441f797b0d None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:41.519726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ba586-84ab-4c9c-9921-cc441f797b0d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:41.520141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ba586-84ab-4c9c-9921-cc441f797b0d None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:41.521655 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 28/55] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:41.527244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38392e4-5aa9-4644-9a41-6f4d59825a2e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:41.527662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38392e4-5aa9-4644-9a41-6f4d59825a2e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:41.527997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38392e4-5aa9-4644-9a41-6f4d59825a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:41.553861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d38392e4-5aa9-4644-9a41-6f4d59825a2e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:41.555067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d38392e4-5aa9-4644-9a41-6f4d59825a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:41.632330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:41.646189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 28/56] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:41.861486 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e20b0e-79a4-4323-808d-80e18aea3dc3 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:41.862122 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e20b0e-79a4-4323-808d-80e18aea3dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:41.862600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e20b0e-79a4-4323-808d-80e18aea3dc3 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:41.864144 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 29/57] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:41.869889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0edbbd-9eab-4dfc-9bed-a09709839d69 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:41.870180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0edbbd-9eab-4dfc-9bed-a09709839d69 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:41.870619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0edbbd-9eab-4dfc-9bed-a09709839d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:41.883607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf431b-562f-4445-8c27-3a2b4a529978 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:41.884313 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf431b-562f-4445-8c27-3a2b4a529978 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:41.884782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf431b-562f-4445-8c27-3a2b4a529978 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:41.886359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 30/58] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:41.890731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-9f0edbbd-9eab-4dfc-9bed-a09709839d69 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:41.892002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace5a95-8aa7-48f5-bdf5-9d9763d77c49 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:41.892297 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9f0edbbd-9eab-4dfc-9bed-a09709839d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:41.892461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace5a95-8aa7-48f5-bdf5-9d9763d77c49 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:41.892856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace5a95-8aa7-48f5-bdf5-9d9763d77c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:41.924784 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3ace5a95-8aa7-48f5-bdf5-9d9763d77c49 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:41.926424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3ace5a95-8aa7-48f5-bdf5-9d9763d77c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:41.971873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 29/59] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:41.981198 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:42.038264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ae657-b079-4697-8f87-a31842249db8 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.038810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ae657-b079-4697-8f87-a31842249db8 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.039225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ae657-b079-4697-8f87-a31842249db8 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.040922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 30/60] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:26:42.046053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f461ed3-f8a3-4f23-9be4-9cd612fea932 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.046620 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f461ed3-f8a3-4f23-9be4-9cd612fea932 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.047070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f461ed3-f8a3-4f23-9be4-9cd612fea932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.058271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 31/61] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:26:42.059202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace5a95-8aa7-48f5-bdf5-9d9763d77c49 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:42.071317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-3f461ed3-f8a3-4f23-9be4-9cd612fea932 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:42.073119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3f461ed3-f8a3-4f23-9be4-9cd612fea932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:42.208489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 31/62] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:26:42.233179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace5a95-8aa7-48f5-bdf5-9d9763d77c49 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:42.289347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f051ce-8a6d-489d-8af2-c7b144a35a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:42.289998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f051ce-8a6d-489d-8af2-c7b144a35a67 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.290275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f051ce-8a6d-489d-8af2-c7b144a35a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.290548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f051ce-8a6d-489d-8af2-c7b144a35a67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.305761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f051ce-8a6d-489d-8af2-c7b144a35a67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-80f051ce-8a6d-489d-8af2-c7b144a35a67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:42.310762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:42.311593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:26:42.312936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 32/63] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:42.319771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f461ed3-f8a3-4f23-9be4-9cd612fea932 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:42.322032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f051ce-8a6d-489d-8af2-c7b144a35a67 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:42.363153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:42.395843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:42.446624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f461ed3-f8a3-4f23-9be4-9cd612fea932 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:42.455739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f051ce-8a6d-489d-8af2-c7b144a35a67 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:42.481523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48e95-7d8e-4351-a1fb-d3f04a4c0d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:42.482167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48e95-7d8e-4351-a1fb-d3f04a4c0d91 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.482449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48e95-7d8e-4351-a1fb-d3f04a4c0d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.482725 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48e95-7d8e-4351-a1fb-d3f04a4c0d91 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.492161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572d30e-864a-4113-9ff2-da7b5497c0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:42.492775 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572d30e-864a-4113-9ff2-da7b5497c0ae None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.493083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572d30e-864a-4113-9ff2-da7b5497c0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.493369 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572d30e-864a-4113-9ff2-da7b5497c0ae None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.494030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572d30e-864a-4113-9ff2-da7b5497c0ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:42.494285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d48e95-7d8e-4351-a1fb-d3f04a4c0d91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:42.494389 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572d30e-864a-4113-9ff2-da7b5497c0ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:42.495708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572d30e-864a-4113-9ff2-da7b5497c0ae None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-46d48e95-7d8e-4351-a1fb-d3f04a4c0d91 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:42.501370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:42.502377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 32/64] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:26:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:42.505524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48e95-7d8e-4351-a1fb-d3f04a4c0d91 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:42.510662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 33/65] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:26:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:42.511803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7b28f-e4b0-4d5e-afde-c542e299216c None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.512091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7b28f-e4b0-4d5e-afde-c542e299216c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.512363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7b28f-e4b0-4d5e-afde-c542e299216c None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.513764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 34/66] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:42.516680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7b28f-e4b0-4d5e-afde-c542e299216c None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:42.599277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:42.604569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48e95-7d8e-4351-a1fb-d3f04a4c0d91 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:42.619115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:42.619964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7b28f-e4b0-4d5e-afde-c542e299216c None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:42.641906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecb635-68b8-4077-b741-26d00fda6b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:42.642543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecb635-68b8-4077-b741-26d00fda6b2f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.642822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecb635-68b8-4077-b741-26d00fda6b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.643095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecb635-68b8-4077-b741-26d00fda6b2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.653237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecb635-68b8-4077-b741-26d00fda6b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:42.655875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e00d9a-dc4b-473b-8b62-e76bc5e2aa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:42.656495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00d9a-dc4b-473b-8b62-e76bc5e2aa15 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.656780 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00d9a-dc4b-473b-8b62-e76bc5e2aa15 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.657090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00d9a-dc4b-473b-8b62-e76bc5e2aa15 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.657757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e00d9a-dc4b-473b-8b62-e76bc5e2aa15 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:42.658064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e00d9a-dc4b-473b-8b62-e76bc5e2aa15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-8eecb635-68b8-4077-b741-26d00fda6b2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:42.658179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:42.659104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:42.659861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 33/67] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:26:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:42.660493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e00d9a-dc4b-473b-8b62-e76bc5e2aa15 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:42.660565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecb635-68b8-4077-b741-26d00fda6b2f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:42.667719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 35/68] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:26:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:42.668539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e00d9a-dc4b-473b-8b62-e76bc5e2aa15 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:42.761546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecb635-68b8-4077-b741-26d00fda6b2f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:42.768239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e00d9a-dc4b-473b-8b62-e76bc5e2aa15 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:42.795055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6491a75-6505-4e68-8702-134730caebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:42.795685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6491a75-6505-4e68-8702-134730caebed None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.795967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6491a75-6505-4e68-8702-134730caebed None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.796238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6491a75-6505-4e68-8702-134730caebed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.801495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa08c4-1aaa-4062-a95a-b37be962d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:42.802133 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa08c4-1aaa-4062-a95a-b37be962d6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.802409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa08c4-1aaa-4062-a95a-b37be962d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.802680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa08c4-1aaa-4062-a95a-b37be962d6c3 None admin] PATH_INFO: `/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.806659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6491a75-6505-4e68-8702-134730caebed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-c6491a75-6505-4e68-8702-134730caebed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:42.811501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:42.812987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:42.813777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 34/69] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:42.814489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d9b0c-df3b-449b-aee3-e24d0d9dcc2e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.814665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d9b0c-df3b-449b-aee3-e24d0d9dcc2e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.814874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d9b0c-df3b-449b-aee3-e24d0d9dcc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.818339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:42.821132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa08c4-1aaa-4062-a95a-b37be962d6c3 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c3b13d60ab475ba56f29d0a183c8c9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:42.823249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa08c4-1aaa-4062-a95a-b37be962d6c3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:42.824952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 36/70] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:42 2022] GET /identity/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:42.825730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa08c4-1aaa-4062-a95a-b37be962d6c3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:42.833481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-b25d9b0c-df3b-449b-aee3-e24d0d9dcc2e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:42.834709 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b25d9b0c-df3b-449b-aee3-e24d0d9dcc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:42.861080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:42.901300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 35/71] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:42.902548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d9b0c-df3b-449b-aee3-e24d0d9dcc2e None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:42.943334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa08c4-1aaa-4062-a95a-b37be962d6c3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:42.978693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f18e0-2152-4451-8ed5-6f6d3008600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:42.979312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f18e0-2152-4451-8ed5-6f6d3008600c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:42.979598 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f18e0-2152-4451-8ed5-6f6d3008600c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:42.979875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f18e0-2152-4451-8ed5-6f6d3008600c None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:42.980604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f18e0-2152-4451-8ed5-6f6d3008600c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:42.980946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f18e0-2152-4451-8ed5-6f6d3008600c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:42.982329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f18e0-2152-4451-8ed5-6f6d3008600c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:42.995332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 37/72] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:42.995332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f18e0-2152-4451-8ed5-6f6d3008600c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.011893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d9b0c-df3b-449b-aee3-e24d0d9dcc2e None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.047086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5f9f9-c80e-41ce-8394-ee80ca787514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.047891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5f9f9-c80e-41ce-8394-ee80ca787514 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.048179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5f9f9-c80e-41ce-8394-ee80ca787514 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.048458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5f9f9-c80e-41ce-8394-ee80ca787514 None admin] PATH_INFO: `/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.049616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5f9f9-c80e-41ce-8394-ee80ca787514 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c3b13d60ab475ba56f29d0a183c8c9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.051071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5f9f9-c80e-41ce-8394-ee80ca787514 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.053099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 36/73] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:42 2022] GET /identity/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:43.053879 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5f9f9-c80e-41ce-8394-ee80ca787514 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.061359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:43.089953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:43.104256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f18e0-2152-4451-8ed5-6f6d3008600c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.140995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf192e01-5b21-411e-8824-d5b7bde6aa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.141623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf192e01-5b21-411e-8824-d5b7bde6aa4c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.141905 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf192e01-5b21-411e-8824-d5b7bde6aa4c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.142183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf192e01-5b21-411e-8824-d5b7bde6aa4c None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.143016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf192e01-5b21-411e-8824-d5b7bde6aa4c None admin] RBAC: Adding query filter params (name=member) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:43.143336 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf192e01-5b21-411e-8824-d5b7bde6aa4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.144909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf192e01-5b21-411e-8824-d5b7bde6aa4c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.158632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 38/74] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:26:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:43.158632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf192e01-5b21-411e-8824-d5b7bde6aa4c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.166441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5f9f9-c80e-41ce-8394-ee80ca787514 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.204646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a7bb-8dec-4268-88b0-62fbf33d2304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.205302 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a7bb-8dec-4268-88b0-62fbf33d2304 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.205584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a7bb-8dec-4268-88b0-62fbf33d2304 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.205856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a7bb-8dec-4268-88b0-62fbf33d2304 None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.206977 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4a7bb-8dec-4268-88b0-62fbf33d2304 None admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.208421 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4a7bb-8dec-4268-88b0-62fbf33d2304 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.210246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 37/75] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:43 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:43.210820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a7bb-8dec-4268-88b0-62fbf33d2304 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.263926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf192e01-5b21-411e-8824-d5b7bde6aa4c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.287296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:43.299570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d0a89-ee5b-4093-94e3-b75bbca06a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.300603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0a89-ee5b-4093-94e3-b75bbca06a36 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.300603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0a89-ee5b-4093-94e3-b75bbca06a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.300811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0a89-ee5b-4093-94e3-b75bbca06a36 None admin] PATH_INFO: `/v3/users/c5f193bd980d483b8200f674f5249745` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.301992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d0a89-ee5b-4093-94e3-b75bbca06a36 None admin] RBAC: Authorizing `identity:get_user(user_id=c5f193bd980d483b8200f674f5249745)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.303479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d0a89-ee5b-4093-94e3-b75bbca06a36 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.305123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 39/76] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:43 2022] GET /identity/v3/users/c5f193bd980d483b8200f674f5249745 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:43.305901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d0a89-ee5b-4093-94e3-b75bbca06a36 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.308782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:43.313452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a7bb-8dec-4268-88b0-62fbf33d2304 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.351716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa2376-eb4f-4b43-a895-41833632113b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.352354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa2376-eb4f-4b43-a895-41833632113b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.352637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa2376-eb4f-4b43-a895-41833632113b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.352948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa2376-eb4f-4b43-a895-41833632113b None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.354089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa2376-eb4f-4b43-a895-41833632113b None admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.355542 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa2376-eb4f-4b43-a895-41833632113b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.357208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 38/77] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:43 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:43.357977 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa2376-eb4f-4b43-a895-41833632113b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.408325 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d0a89-ee5b-4093-94e3-b75bbca06a36 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.441912 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbd75b-26af-4c6f-936e-4aed051385a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.442560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbd75b-26af-4c6f-936e-4aed051385a3 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.442841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbd75b-26af-4c6f-936e-4aed051385a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.443115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbd75b-26af-4c6f-936e-4aed051385a3 None admin] PATH_INFO: `/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.444265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbd75b-26af-4c6f-936e-4aed051385a3 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c3b13d60ab475ba56f29d0a183c8c9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.445758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbd75b-26af-4c6f-936e-4aed051385a3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.447336 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 40/78] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:43 2022] GET /identity/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:43.452062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbd75b-26af-4c6f-936e-4aed051385a3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.458274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa2376-eb4f-4b43-a895-41833632113b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.491705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4699-1415-4f0c-b006-b5fdf4aa2bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.492345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4699-1415-4f0c-b006-b5fdf4aa2bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.492633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4699-1415-4f0c-b006-b5fdf4aa2bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.492947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4699-1415-4f0c-b006-b5fdf4aa2bd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.494076 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a4699-1415-4f0c-b006-b5fdf4aa2bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.495522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a4699-1415-4f0c-b006-b5fdf4aa2bd4 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.497126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 39/79] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:43.497915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4699-1415-4f0c-b006-b5fdf4aa2bd4 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.510598 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:43.533154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:43.551093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbd75b-26af-4c6f-936e-4aed051385a3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.584592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d0283-f2fa-451b-a52a-787d11ee6f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.585285 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d0283-f2fa-451b-a52a-787d11ee6f1e None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.585568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d0283-f2fa-451b-a52a-787d11ee6f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.585845 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d0283-f2fa-451b-a52a-787d11ee6f1e None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/users/e9c3b13d60ab475ba56f29d0a183c8c9/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.587769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d0283-f2fa-451b-a52a-787d11ee6f1e None admin] RBAC: Authorizing `identity:create_grant(project_id=ba4a0bd34a554747b58ff7dc25238f38, user_id=e9c3b13d60ab475ba56f29d0a183c8c9, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.589239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d0283-f2fa-451b-a52a-787d11ee6f1e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.600068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4699-1415-4f0c-b006-b5fdf4aa2bd4 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.602051 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 41/80] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:26:43 2022] PUT /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/users/e9c3b13d60ab475ba56f29d0a183c8c9/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:26:43.602831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d0283-f2fa-451b-a52a-787d11ee6f1e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.641769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc45a0f-600b-41d6-9d02-f8559873fdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.642463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc45a0f-600b-41d6-9d02-f8559873fdae None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.642698 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc45a0f-600b-41d6-9d02-f8559873fdae None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.642972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc45a0f-600b-41d6-9d02-f8559873fdae None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.644185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc45a0f-600b-41d6-9d02-f8559873fdae None admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.645690 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc45a0f-600b-41d6-9d02-f8559873fdae None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.647395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 40/81] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:43 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:43.649818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc45a0f-600b-41d6-9d02-f8559873fdae None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.720233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d0283-f2fa-451b-a52a-787d11ee6f1e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.741934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:43.758818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b78ae-f261-4f82-9b47-805630fe8bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.759597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b78ae-f261-4f82-9b47-805630fe8bc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.759969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b78ae-f261-4f82-9b47-805630fe8bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.760309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b78ae-f261-4f82-9b47-805630fe8bc2 None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/users/e9c3b13d60ab475ba56f29d0a183c8c9/roles/04b8e62238cb4198928fd327ef78e6ae` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.762728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b78ae-f261-4f82-9b47-805630fe8bc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba4a0bd34a554747b58ff7dc25238f38, user_id=e9c3b13d60ab475ba56f29d0a183c8c9, role_id=04b8e62238cb4198928fd327ef78e6ae)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.764304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b78ae-f261-4f82-9b47-805630fe8bc2 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.777301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc45a0f-600b-41d6-9d02-f8559873fdae None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.779408 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:43.784944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 42/82] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:26:43 2022] PUT /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/users/e9c3b13d60ab475ba56f29d0a183c8c9/roles/04b8e62238cb4198928fd327ef78e6ae => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:26:43.784944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b78ae-f261-4f82-9b47-805630fe8bc2 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.823235 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2ec5d-9721-4b16-93df-7beaad21e96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.824047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2ec5d-9721-4b16-93df-7beaad21e96d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.824789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2ec5d-9721-4b16-93df-7beaad21e96d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.824789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2ec5d-9721-4b16-93df-7beaad21e96d None admin] PATH_INFO: `/v3/projects/dbf9b59792db4112935a6d574c00be70` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.826101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2ec5d-9721-4b16-93df-7beaad21e96d None admin] RBAC: Authorizing `identity:get_project(project_id=dbf9b59792db4112935a6d574c00be70)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.827666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2ec5d-9721-4b16-93df-7beaad21e96d None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:43.831375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 41/83] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:43 2022] GET /identity/v3/projects/dbf9b59792db4112935a6d574c00be70 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:43.836052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2ec5d-9721-4b16-93df-7beaad21e96d None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:43.943888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b78ae-f261-4f82-9b47-805630fe8bc2 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.966932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2ec5d-9721-4b16-93df-7beaad21e96d None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:43.992062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578fd6-afae-40bc-8282-89fc46872766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:43.993762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578fd6-afae-40bc-8282-89fc46872766 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:43.994267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578fd6-afae-40bc-8282-89fc46872766 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:43.994983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578fd6-afae-40bc-8282-89fc46872766 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:43.996119 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578fd6-afae-40bc-8282-89fc46872766 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:43.998551 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578fd6-afae-40bc-8282-89fc46872766 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:44.008312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5cb6e-cc4d-49ef-a830-694c19b43207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:44.009023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5cb6e-cc4d-49ef-a830-694c19b43207 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:44.009316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5cb6e-cc4d-49ef-a830-694c19b43207 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:44.009597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5cb6e-cc4d-49ef-a830-694c19b43207 None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/users/c5f193bd980d483b8200f674f5249745/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:44.011605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5cb6e-cc4d-49ef-a830-694c19b43207 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba4a0bd34a554747b58ff7dc25238f38, user_id=c5f193bd980d483b8200f674f5249745, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:44.013116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5cb6e-cc4d-49ef-a830-694c19b43207 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:44.031907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 42/84] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:26:43 2022] PUT /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/users/c5f193bd980d483b8200f674f5249745/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:26:44.037103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5cb6e-cc4d-49ef-a830-694c19b43207 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:44.045322 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 43/85] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:26:43 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:44.078283 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:44.188922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5cb6e-cc4d-49ef-a830-694c19b43207 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:44.245170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bcb96-0702-4bee-9bcb-e309b18cae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:44.245917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bcb96-0702-4bee-9bcb-e309b18cae50 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:44.246206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bcb96-0702-4bee-9bcb-e309b18cae50 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:44.246526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bcb96-0702-4bee-9bcb-e309b18cae50 None admin] PATH_INFO: `/v3/projects/dbf9b59792db4112935a6d574c00be70/users/e9c3b13d60ab475ba56f29d0a183c8c9/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:44.248582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bcb96-0702-4bee-9bcb-e309b18cae50 None admin] RBAC: Authorizing `identity:create_grant(project_id=dbf9b59792db4112935a6d574c00be70, user_id=e9c3b13d60ab475ba56f29d0a183c8c9, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:44.250141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bcb96-0702-4bee-9bcb-e309b18cae50 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:44.283066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 43/86] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:26:44 2022] PUT /identity/v3/projects/dbf9b59792db4112935a6d574c00be70/users/e9c3b13d60ab475ba56f29d0a183c8c9/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:26:46.127217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3efd3-3a98-47b1-b1c5-15833d50e97c None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:46.127789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3efd3-3a98-47b1-b1c5-15833d50e97c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:46.127789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3efd3-3a98-47b1-b1c5-15833d50e97c None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:46.129242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 44/87] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:46.135678 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb1185-d7e9-42e5-8265-4c7f197daa86 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:46.136189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb1185-d7e9-42e5-8265-4c7f197daa86 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:46.136189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb1185-d7e9-42e5-8265-4c7f197daa86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:46.165946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-b0fb1185-d7e9-42e5-8265-4c7f197daa86 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:46.167189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b0fb1185-d7e9-42e5-8265-4c7f197daa86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:46.257315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 44/88] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:46.308052 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c5413-0f19-4f71-981a-838114d93afe None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:46.308348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c5413-0f19-4f71-981a-838114d93afe None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:46.308611 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c5413-0f19-4f71-981a-838114d93afe None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:46.310244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 45/89] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:46.316821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ce4f6-d0a1-42c0-b6a4-b2bbd975a5ee None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:46.317143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ce4f6-d0a1-42c0-b6a4-b2bbd975a5ee None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:46.317428 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ce4f6-d0a1-42c0-b6a4-b2bbd975a5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:46.345951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-a93ce4f6-d0a1-42c0-b6a4-b2bbd975a5ee None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:46.347172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a93ce4f6-d0a1-42c0-b6a4-b2bbd975a5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:46.363569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:46.439775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 45/90] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:46.712216 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c5413-0f19-4f71-981a-838114d93afe None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:46.760866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:46.844999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c5413-0f19-4f71-981a-838114d93afe None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:46.893239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d9d9d-cc53-40be-aee1-bbf0ea8557d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:46.893913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d9d9d-cc53-40be-aee1-bbf0ea8557d8 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:46.894191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d9d9d-cc53-40be-aee1-bbf0ea8557d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:46.894465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d9d9d-cc53-40be-aee1-bbf0ea8557d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:46.908627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d9d9d-cc53-40be-aee1-bbf0ea8557d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-e31d9d9d-cc53-40be-aee1-bbf0ea8557d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:46.913672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:46.914470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:26:46.916011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 46/91] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:46.926022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ce4f6-d0a1-42c0-b6a4-b2bbd975a5ee None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:46.995187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d9d9d-cc53-40be-aee1-bbf0ea8557d8 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:47.107869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ce4f6-d0a1-42c0-b6a4-b2bbd975a5ee None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:47.116003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:47.127951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d9d9d-cc53-40be-aee1-bbf0ea8557d8 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:47.172484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e668c09-3dd7-4f96-a2f4-1eb558cf8b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:47.173185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e668c09-3dd7-4f96-a2f4-1eb558cf8b98 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:47.173461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e668c09-3dd7-4f96-a2f4-1eb558cf8b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:47.173730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e668c09-3dd7-4f96-a2f4-1eb558cf8b98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:47.179811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643466b7-472d-414e-b2d1-60d6ce98f4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:47.179811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643466b7-472d-414e-b2d1-60d6ce98f4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:47.180400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643466b7-472d-414e-b2d1-60d6ce98f4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:47.180400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643466b7-472d-414e-b2d1-60d6ce98f4b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:47.181126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643466b7-472d-414e-b2d1-60d6ce98f4b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:47.181413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643466b7-472d-414e-b2d1-60d6ce98f4b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:47.188873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643466b7-472d-414e-b2d1-60d6ce98f4b1 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:47.204249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e668c09-3dd7-4f96-a2f4-1eb558cf8b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-9e668c09-3dd7-4f96-a2f4-1eb558cf8b98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:47.209597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:47.210474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:26:47.211630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 47/92] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:26:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:26:47.212244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9392d68-a396-4d41-a346-9f8e00872977 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:47.212244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9392d68-a396-4d41-a346-9f8e00872977 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:47.212618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9392d68-a396-4d41-a346-9f8e00872977 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:47.214407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 48/93] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:47.217865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 46/94] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:26:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:47.220007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288b659-d83d-450e-b8e2-48cb1af38a31 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:47.220353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288b659-d83d-450e-b8e2-48cb1af38a31 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:47.220690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288b659-d83d-450e-b8e2-48cb1af38a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:47.224649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643466b7-472d-414e-b2d1-60d6ce98f4b1 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:47.250469 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:47.262977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9288b659-d83d-450e-b8e2-48cb1af38a31 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:47.264210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9288b659-d83d-450e-b8e2-48cb1af38a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:47.346182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 49/95] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:47.347315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288b659-d83d-450e-b8e2-48cb1af38a31 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:47.369029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643466b7-472d-414e-b2d1-60d6ce98f4b1 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:47.438416 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:47.453793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288b659-d83d-450e-b8e2-48cb1af38a31 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:47.469913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27075580-693d-48a7-964a-89710841e38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:47.470575 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27075580-693d-48a7-964a-89710841e38b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:47.470859 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27075580-693d-48a7-964a-89710841e38b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:47.471130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27075580-693d-48a7-964a-89710841e38b None admin] PATH_INFO: `/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:47.472465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27075580-693d-48a7-964a-89710841e38b None admin] RBAC: Authorizing `identity:get_user(user_id=e9c3b13d60ab475ba56f29d0a183c8c9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:47.474091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27075580-693d-48a7-964a-89710841e38b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:47.484883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 47/96] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:47 2022] GET /identity/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:26:47.484883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5626-142d-48cb-8244-63ae84535932 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:47.484883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5626-142d-48cb-8244-63ae84535932 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:47.484883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5626-142d-48cb-8244-63ae84535932 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:47.484883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 48/97] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:47.484883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc5626-142d-48cb-8244-63ae84535932 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:47.492245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ef28d-6f36-4912-bbc9-044ddbe1d8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:47.493105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ef28d-6f36-4912-bbc9-044ddbe1d8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:47.493547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ef28d-6f36-4912-bbc9-044ddbe1d8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:47.493953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ef28d-6f36-4912-bbc9-044ddbe1d8a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:47.494755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ef28d-6f36-4912-bbc9-044ddbe1d8a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:47.495193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ef28d-6f36-4912-bbc9-044ddbe1d8a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:47.496711 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ef28d-6f36-4912-bbc9-044ddbe1d8a9 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:47.506270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 50/98] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:26:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:47.507758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a1207-cd90-4925-bcf9-b1bff11d64ea None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:47.508203 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a1207-cd90-4925-bcf9-b1bff11d64ea None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:47.508615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a1207-cd90-4925-bcf9-b1bff11d64ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:47.527483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-768a1207-cd90-4925-bcf9-b1bff11d64ea None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:47.528954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-768a1207-cd90-4925-bcf9-b1bff11d64ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:47.639459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 51/99] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:47.641214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a1207-cd90-4925-bcf9-b1bff11d64ea None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:47.649359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:47.684163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc5626-142d-48cb-8244-63ae84535932 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:47.769672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d174a24-5fbb-4eb7-95d5-401a67b4378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:47.770754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d174a24-5fbb-4eb7-95d5-401a67b4378b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:47.771765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d174a24-5fbb-4eb7-95d5-401a67b4378b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:47.772183 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d174a24-5fbb-4eb7-95d5-401a67b4378b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:47.784950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:47.834111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a1207-cd90-4925-bcf9-b1bff11d64ea None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:47.879806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d174a24-5fbb-4eb7-95d5-401a67b4378b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:47.881329 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d174a24-5fbb-4eb7-95d5-401a67b4378b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:47.885320 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8470ca8-3e44-4d66-97b9-85cf0ea965bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:47.886088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8470ca8-3e44-4d66-97b9-85cf0ea965bd None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:47.886449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8470ca8-3e44-4d66-97b9-85cf0ea965bd None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:47.886796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8470ca8-3e44-4d66-97b9-85cf0ea965bd None admin] PATH_INFO: `/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:47.888032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8470ca8-3e44-4d66-97b9-85cf0ea965bd None admin] RBAC: Authorizing `identity:get_user(user_id=e9c3b13d60ab475ba56f29d0a183c8c9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:47.890214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8470ca8-3e44-4d66-97b9-85cf0ea965bd None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:47.892692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 52/100] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:47 2022] GET /identity/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:47.893567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8470ca8-3e44-4d66-97b9-85cf0ea965bd None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:47.974260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:47.997641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 49/101] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:26:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:26:48.003996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeb897-2341-47ee-bc9b-bc4c09f0f84c None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.004318 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeb897-2341-47ee-bc9b-bc4c09f0f84c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.004597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeb897-2341-47ee-bc9b-bc4c09f0f84c None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.006097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 50/102] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:48.010607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaeb897-2341-47ee-bc9b-bc4c09f0f84c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:48.030455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8470ca8-3e44-4d66-97b9-85cf0ea965bd None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:48.066915 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4867c9-1653-4b9e-a69d-dc52fd7f248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:48.067563 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4867c9-1653-4b9e-a69d-dc52fd7f248a None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.067844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4867c9-1653-4b9e-a69d-dc52fd7f248a None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.068118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4867c9-1653-4b9e-a69d-dc52fd7f248a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.080713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4867c9-1653-4b9e-a69d-dc52fd7f248a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-1c4867c9-1653-4b9e-a69d-dc52fd7f248a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:48.086266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:48.087193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:26:48.087937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 53/103] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:26:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:48.088566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4867c9-1653-4b9e-a69d-dc52fd7f248a None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:48.099306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:48.128978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaeb897-2341-47ee-bc9b-bc4c09f0f84c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:48.166606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b907ef-fc6e-41f0-839f-0c0342dd240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:48.166606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b907ef-fc6e-41f0-839f-0c0342dd240c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.166606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b907ef-fc6e-41f0-839f-0c0342dd240c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.166606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b907ef-fc6e-41f0-839f-0c0342dd240c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.205896 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4867c9-1653-4b9e-a69d-dc52fd7f248a None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:48.222448 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:48.240615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66d2a-98f4-4261-a6dc-47fd26e371f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:48.241283 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66d2a-98f4-4261-a6dc-47fd26e371f5 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.241564 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66d2a-98f4-4261-a6dc-47fd26e371f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.241836 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66d2a-98f4-4261-a6dc-47fd26e371f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.242522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b907ef-fc6e-41f0-839f-0c0342dd240c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:48.244053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b907ef-fc6e-41f0-839f-0c0342dd240c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:48.252073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c66d2a-98f4-4261-a6dc-47fd26e371f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-90c66d2a-98f4-4261-a6dc-47fd26e371f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:48.257440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:48.258476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:26:48.259302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 54/104] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:48.259951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66d2a-98f4-4261-a6dc-47fd26e371f5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:48.325495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:48.341394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 51/105] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:26:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:26:48.342648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daeb33-bc10-4418-ae51-5d9b4df02d0b None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.342936 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daeb33-bc10-4418-ae51-5d9b4df02d0b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.343785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2daeb33-bc10-4418-ae51-5d9b4df02d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.360804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66d2a-98f4-4261-a6dc-47fd26e371f5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:48.361955 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e2daeb33-bc10-4418-ae51-5d9b4df02d0b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:48.363167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e2daeb33-bc10-4418-ae51-5d9b4df02d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:48.395202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbcaa0-f4b1-4712-9b6d-27f7f5a44943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:48.395837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbcaa0-f4b1-4712-9b6d-27f7f5a44943 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.396119 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbcaa0-f4b1-4712-9b6d-27f7f5a44943 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.396392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbcaa0-f4b1-4712-9b6d-27f7f5a44943 None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.397497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbcaa0-f4b1-4712-9b6d-27f7f5a44943 None admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:48.398979 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbcaa0-f4b1-4712-9b6d-27f7f5a44943 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:48.400630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 55/106] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:48 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:48.401433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbcaa0-f4b1-4712-9b6d-27f7f5a44943 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:48.433229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 52/107] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:48.434302 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce404bc-2698-4876-b336-175869752a08 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.434586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce404bc-2698-4876-b336-175869752a08 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.434862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce404bc-2698-4876-b336-175869752a08 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.436098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 53/108] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:48.436794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce404bc-2698-4876-b336-175869752a08 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:48.449054 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:48.511030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbcaa0-f4b1-4712-9b6d-27f7f5a44943 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:48.542811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce404bc-2698-4876-b336-175869752a08 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:48.551749 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6b12f-d84d-48c1-9c13-38b336479112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:48.552374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6b12f-d84d-48c1-9c13-38b336479112 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.552652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6b12f-d84d-48c1-9c13-38b336479112 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.552951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6b12f-d84d-48c1-9c13-38b336479112 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.553645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6b12f-d84d-48c1-9c13-38b336479112 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:48.553947 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6b12f-d84d-48c1-9c13-38b336479112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:48.555335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6b12f-d84d-48c1-9c13-38b336479112 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:48.557031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:48.565385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 56/109] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:48.566210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6b12f-d84d-48c1-9c13-38b336479112 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:48.585732 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4e0b9-c7d4-46cb-9597-a7bb3334ac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:48.587020 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4e0b9-c7d4-46cb-9597-a7bb3334ac3b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.587465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4e0b9-c7d4-46cb-9597-a7bb3334ac3b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.587750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4e0b9-c7d4-46cb-9597-a7bb3334ac3b None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.588423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4e0b9-c7d4-46cb-9597-a7bb3334ac3b None admin] RBAC: Adding query filter params (name=member) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:48.588730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4e0b9-c7d4-46cb-9597-a7bb3334ac3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:48.590123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4e0b9-c7d4-46cb-9597-a7bb3334ac3b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:48.598989 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 54/110] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:26:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:48.599795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4e0b9-c7d4-46cb-9597-a7bb3334ac3b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:48.674324 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:48.687421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6b12f-d84d-48c1-9c13-38b336479112 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:48.708780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4e0b9-c7d4-46cb-9597-a7bb3334ac3b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:48.723380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321e9d8-a913-4ac8-ab45-0ecca14726a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:48.724006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321e9d8-a913-4ac8-ab45-0ecca14726a7 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.724280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321e9d8-a913-4ac8-ab45-0ecca14726a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.724549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321e9d8-a913-4ac8-ab45-0ecca14726a7 None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.725680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321e9d8-a913-4ac8-ab45-0ecca14726a7 None admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:48.727132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321e9d8-a913-4ac8-ab45-0ecca14726a7 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:48.728742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 57/111] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:48 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:48.732431 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d0ca6-76ac-49fc-bc8e-00f3f6ddc45c None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.732718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d0ca6-76ac-49fc-bc8e-00f3f6ddc45c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.733039 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d0ca6-76ac-49fc-bc8e-00f3f6ddc45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.743912 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e40bba-b9f3-4e9c-af6e-29e8168f47ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:48.744524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e40bba-b9f3-4e9c-af6e-29e8168f47ed None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.744808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e40bba-b9f3-4e9c-af6e-29e8168f47ed None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.745109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e40bba-b9f3-4e9c-af6e-29e8168f47ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.745848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e40bba-b9f3-4e9c-af6e-29e8168f47ed None admin] RBAC: Adding query filter params (role.id=846f40e28e104ec4bc568fe533f9c9a9, scope.project.id=ba4a0bd34a554747b58ff7dc25238f38, user.id=e9c3b13d60ab475ba56f29d0a183c8c9) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:48.746161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e40bba-b9f3-4e9c-af6e-29e8168f47ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:48.747598 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e40bba-b9f3-4e9c-af6e-29e8168f47ed None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:48.751033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4d7d0ca6-76ac-49fc-bc8e-00f3f6ddc45c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:48.752238 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4d7d0ca6-76ac-49fc-bc8e-00f3f6ddc45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:48.753518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 55/112] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:26:48 2022] GET /identity/v3/role_assignments?user.id=e9c3b13d60ab475ba56f29d0a183c8c9&scope.project.id=ba4a0bd34a554747b58ff7dc25238f38&role.id=846f40e28e104ec4bc568fe533f9c9a9 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:48.754492 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e40bba-b9f3-4e9c-af6e-29e8168f47ed None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:48.823988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:48.834637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 58/113] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:48.842158 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d0ca6-76ac-49fc-bc8e-00f3f6ddc45c None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:48.880171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e40bba-b9f3-4e9c-af6e-29e8168f47ed None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:48.930264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f5efa-819c-40f5-b586-e0a53c82aa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:48.931911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f5efa-819c-40f5-b586-e0a53c82aa85 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:48.933708 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f5efa-819c-40f5-b586-e0a53c82aa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:48.933708 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f5efa-819c-40f5-b586-e0a53c82aa85 None admin] PATH_INFO: `/v3/users/c5f193bd980d483b8200f674f5249745` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:48.936246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f5efa-819c-40f5-b586-e0a53c82aa85 None admin] RBAC: Authorizing `identity:get_user(user_id=c5f193bd980d483b8200f674f5249745)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:48.936578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f5efa-819c-40f5-b586-e0a53c82aa85 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:48.939097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 56/114] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:48 2022] GET /identity/v3/users/c5f193bd980d483b8200f674f5249745 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:48.939638 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f5efa-819c-40f5-b586-e0a53c82aa85 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:48.944296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:48.990801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d0ca6-76ac-49fc-bc8e-00f3f6ddc45c None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.034762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545c6c6-1e29-41ff-ae20-c65e158b9e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.035445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c6c6-1e29-41ff-ae20-c65e158b9e97 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.035729 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c6c6-1e29-41ff-ae20-c65e158b9e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.036003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c6c6-1e29-41ff-ae20-c65e158b9e97 None admin] PATH_INFO: `/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:49.037325 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545c6c6-1e29-41ff-ae20-c65e158b9e97 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c3b13d60ab475ba56f29d0a183c8c9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:49.038839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545c6c6-1e29-41ff-ae20-c65e158b9e97 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:49.040602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 59/115] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:48 2022] GET /identity/v3/users/e9c3b13d60ab475ba56f29d0a183c8c9 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:49.045637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545c6c6-1e29-41ff-ae20-c65e158b9e97 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:49.073947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f5efa-819c-40f5-b586-e0a53c82aa85 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.110422 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:49.113427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7373a06-ded3-44fb-9cd0-d117718243f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.114072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7373a06-ded3-44fb-9cd0-d117718243f0 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.114348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7373a06-ded3-44fb-9cd0-d117718243f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.114877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7373a06-ded3-44fb-9cd0-d117718243f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:49.125227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7373a06-ded3-44fb-9cd0-d117718243f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-d7373a06-ded3-44fb-9cd0-d117718243f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:49.130468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:49.131314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:49.132255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 57/116] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:49.133263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7373a06-ded3-44fb-9cd0-d117718243f0 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:49.152518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545c6c6-1e29-41ff-ae20-c65e158b9e97 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.188707 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ec73c-4199-467b-a2c9-f28a8ae1e398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.189355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ec73c-4199-467b-a2c9-f28a8ae1e398 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.189646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ec73c-4199-467b-a2c9-f28a8ae1e398 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.189917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ec73c-4199-467b-a2c9-f28a8ae1e398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:49.190644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ec73c-4199-467b-a2c9-f28a8ae1e398 None admin] RBAC: Adding query filter params (role.id=04b8e62238cb4198928fd327ef78e6ae, scope.project.id=ba4a0bd34a554747b58ff7dc25238f38, user.id=e9c3b13d60ab475ba56f29d0a183c8c9) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:49.190952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ec73c-4199-467b-a2c9-f28a8ae1e398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:49.192390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ec73c-4199-467b-a2c9-f28a8ae1e398 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:49.199403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 60/117] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:26:49 2022] GET /identity/v3/role_assignments?user.id=e9c3b13d60ab475ba56f29d0a183c8c9&scope.project.id=ba4a0bd34a554747b58ff7dc25238f38&role.id=04b8e62238cb4198928fd327ef78e6ae => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:49.200210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ec73c-4199-467b-a2c9-f28a8ae1e398 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:49.226592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:49.271309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7373a06-ded3-44fb-9cd0-d117718243f0 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.312327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e50e9-2213-46bd-8061-6855835a9703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.313065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e50e9-2213-46bd-8061-6855835a9703 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.313350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e50e9-2213-46bd-8061-6855835a9703 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.313624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e50e9-2213-46bd-8061-6855835a9703 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:49.321083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ec73c-4199-467b-a2c9-f28a8ae1e398 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.351023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:49.362191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b772-e39f-4ed1-a2d2-2bcc91b916ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.362748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b772-e39f-4ed1-a2d2-2bcc91b916ab None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.363033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b772-e39f-4ed1-a2d2-2bcc91b916ab None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.363317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b772-e39f-4ed1-a2d2-2bcc91b916ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:49.402394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e50e9-2213-46bd-8061-6855835a9703 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:49.404228 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e50e9-2213-46bd-8061-6855835a9703 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:49.442929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2b772-e39f-4ed1-a2d2-2bcc91b916ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:49.444527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2b772-e39f-4ed1-a2d2-2bcc91b916ab None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:49.471281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:49.510090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 58/118] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:26:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:26:49.511199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e50e9-2213-46bd-8061-6855835a9703 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:49.549829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 61/119] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:26:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:26:49.550702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b772-e39f-4ed1-a2d2-2bcc91b916ab None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:49.581190 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:49.609704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e50e9-2213-46bd-8061-6855835a9703 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.644167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3218f-5157-41ee-9b27-aa3a60ea3276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.644806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3218f-5157-41ee-9b27-aa3a60ea3276 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.645118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3218f-5157-41ee-9b27-aa3a60ea3276 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.645391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3218f-5157-41ee-9b27-aa3a60ea3276 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:49.646503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3218f-5157-41ee-9b27-aa3a60ea3276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:49.647998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3218f-5157-41ee-9b27-aa3a60ea3276 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:49.649589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 59/120] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:49.650358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3218f-5157-41ee-9b27-aa3a60ea3276 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:49.654002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b772-e39f-4ed1-a2d2-2bcc91b916ab None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.680451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:49.689247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1fa81-14a6-48e1-a36d-22c3b604d7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.689868 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1fa81-14a6-48e1-a36d-22c3b604d7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.690148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1fa81-14a6-48e1-a36d-22c3b604d7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1fa81-14a6-48e1-a36d-22c3b604d7b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:49.691108 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1fa81-14a6-48e1-a36d-22c3b604d7b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:49.691401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1fa81-14a6-48e1-a36d-22c3b604d7b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:49.692800 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1fa81-14a6-48e1-a36d-22c3b604d7b1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:49.701125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 62/121] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:26:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:49.702031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1fa81-14a6-48e1-a36d-22c3b604d7b1 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:49.755114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3218f-5157-41ee-9b27-aa3a60ea3276 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.790119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448f9fb-7b30-4296-8c5d-418ef90cc627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.790752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448f9fb-7b30-4296-8c5d-418ef90cc627 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.791038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448f9fb-7b30-4296-8c5d-418ef90cc627 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.791298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:49.791404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448f9fb-7b30-4296-8c5d-418ef90cc627 None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:49.792434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448f9fb-7b30-4296-8c5d-418ef90cc627 None admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:49.793956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448f9fb-7b30-4296-8c5d-418ef90cc627 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:49.795596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 60/122] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:49 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:49.796379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448f9fb-7b30-4296-8c5d-418ef90cc627 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:49.807710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1fa81-14a6-48e1-a36d-22c3b604d7b1 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.842411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec3b7f-14db-4897-ae4b-3accc2f26395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.843058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec3b7f-14db-4897-ae4b-3accc2f26395 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.843344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec3b7f-14db-4897-ae4b-3accc2f26395 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.843623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec3b7f-14db-4897-ae4b-3accc2f26395 None admin] PATH_INFO: `/v3/projects/dbf9b59792db4112935a6d574c00be70` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:49.844764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec3b7f-14db-4897-ae4b-3accc2f26395 None admin] RBAC: Authorizing `identity:get_project(project_id=dbf9b59792db4112935a6d574c00be70)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:49.846257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec3b7f-14db-4897-ae4b-3accc2f26395 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:49.847920 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 63/123] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:49 2022] GET /identity/v3/projects/dbf9b59792db4112935a6d574c00be70 => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:49.848740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec3b7f-14db-4897-ae4b-3accc2f26395 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:49.895733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:49.898801 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448f9fb-7b30-4296-8c5d-418ef90cc627 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.933385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a03322-25ea-4bfd-9a39-66a8e11e99a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.934000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a03322-25ea-4bfd-9a39-66a8e11e99a3 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.934284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a03322-25ea-4bfd-9a39-66a8e11e99a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.934557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a03322-25ea-4bfd-9a39-66a8e11e99a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:49.935217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a03322-25ea-4bfd-9a39-66a8e11e99a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:49.936656 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a03322-25ea-4bfd-9a39-66a8e11e99a3 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:49.950241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec3b7f-14db-4897-ae4b-3accc2f26395 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:49.966213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 61/124] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:26:49 2022] POST /identity/v3/projects => generated 302 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:49.967053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a03322-25ea-4bfd-9a39-66a8e11e99a3 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:49.987507 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f6e66-d16e-4e53-987c-61c698c89dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:49.988163 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f6e66-d16e-4e53-987c-61c698c89dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:49.988441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f6e66-d16e-4e53-987c-61c698c89dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:49.988709 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f6e66-d16e-4e53-987c-61c698c89dc6 None admin] PATH_INFO: `/v3/users/86762b83e1d04cbfbc95d8f083beca07` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:50.005456 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f6e66-d16e-4e53-987c-61c698c89dc6 None admin] RBAC: Authorizing `identity:get_user(user_id=86762b83e1d04cbfbc95d8f083beca07)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:50.006956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f6e66-d16e-4e53-987c-61c698c89dc6 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:50.008994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 64/125] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:49 2022] GET /identity/v3/users/86762b83e1d04cbfbc95d8f083beca07 => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:50.009624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f6e66-d16e-4e53-987c-61c698c89dc6 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:50.027316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:50.092382 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a03322-25ea-4bfd-9a39-66a8e11e99a3 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:50.128579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f6e66-d16e-4e53-987c-61c698c89dc6 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:50.130561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d153e94-3380-4d37-90fc-5760e4412424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:50.131182 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d153e94-3380-4d37-90fc-5760e4412424 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:50.131468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d153e94-3380-4d37-90fc-5760e4412424 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:50.131748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d153e94-3380-4d37-90fc-5760e4412424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:50.132492 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d153e94-3380-4d37-90fc-5760e4412424 None admin] RBAC: Adding query filter params (role.id=8d9ff9da6ad946a6bc49336777182ff9, scope.project.id=ba4a0bd34a554747b58ff7dc25238f38, user.id=c5f193bd980d483b8200f674f5249745) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:50.132806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d153e94-3380-4d37-90fc-5760e4412424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:50.134352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d153e94-3380-4d37-90fc-5760e4412424 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:50.141826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 62/126] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:26:49 2022] GET /identity/v3/role_assignments?user.id=c5f193bd980d483b8200f674f5249745&scope.project.id=ba4a0bd34a554747b58ff7dc25238f38&role.id=8d9ff9da6ad946a6bc49336777182ff9 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:50.142684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d153e94-3380-4d37-90fc-5760e4412424 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:50.163535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:50.176073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee6d61-608d-4ad8-8ca0-684b6328204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:50.176863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee6d61-608d-4ad8-8ca0-684b6328204f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:50.177212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee6d61-608d-4ad8-8ca0-684b6328204f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:50.177558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee6d61-608d-4ad8-8ca0-684b6328204f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:50.178408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee6d61-608d-4ad8-8ca0-684b6328204f None admin] RBAC: Adding query filter params (role.id=846f40e28e104ec4bc568fe533f9c9a9, scope.project.id=dbf9b59792db4112935a6d574c00be70, user.id=e9c3b13d60ab475ba56f29d0a183c8c9) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:50.178732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee6d61-608d-4ad8-8ca0-684b6328204f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:50.180563 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee6d61-608d-4ad8-8ca0-684b6328204f None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:50.189969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 65/127] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:26:50 2022] GET /identity/v3/role_assignments?user.id=e9c3b13d60ab475ba56f29d0a183c8c9&scope.project.id=dbf9b59792db4112935a6d574c00be70&role.id=846f40e28e104ec4bc568fe533f9c9a9 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:50.289958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d153e94-3380-4d37-90fc-5760e4412424 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:50.332528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d5cb8-0fb3-4003-b6e7-5d5ab86ec86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:50.333255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d5cb8-0fb3-4003-b6e7-5d5ab86ec86b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:50.333537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d5cb8-0fb3-4003-b6e7-5d5ab86ec86b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:50.333814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d5cb8-0fb3-4003-b6e7-5d5ab86ec86b None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:50.335064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d5cb8-0fb3-4003-b6e7-5d5ab86ec86b None admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:50.336649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d5cb8-0fb3-4003-b6e7-5d5ab86ec86b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:50.338853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 63/128] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:50 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:50.344451 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee6d61-608d-4ad8-8ca0-684b6328204f None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:50.449353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:50.472820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee6d61-608d-4ad8-8ca0-684b6328204f None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:50.506573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a8229-a9f5-4923-8270-5ca5fb818b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:50.507242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a8229-a9f5-4923-8270-5ca5fb818b0d None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:50.507524 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a8229-a9f5-4923-8270-5ca5fb818b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:50.507801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a8229-a9f5-4923-8270-5ca5fb818b0d None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/users/86762b83e1d04cbfbc95d8f083beca07/roles/235abd0f332d45e69a86877b4fb95973` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:50.509763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a8229-a9f5-4923-8270-5ca5fb818b0d None admin] RBAC: Authorizing `identity:create_grant(project_id=ba4a0bd34a554747b58ff7dc25238f38, user_id=86762b83e1d04cbfbc95d8f083beca07, role_id=235abd0f332d45e69a86877b4fb95973)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:50.511212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a8229-a9f5-4923-8270-5ca5fb818b0d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:50.523730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 66/129] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:26:50 2022] PUT /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/users/86762b83e1d04cbfbc95d8f083beca07/roles/235abd0f332d45e69a86877b4fb95973 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:26:52.219211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c273e52-7bc5-4a47-8ba1-b91e41775aee None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:52.219719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c273e52-7bc5-4a47-8ba1-b91e41775aee None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:52.219827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c273e52-7bc5-4a47-8ba1-b91e41775aee None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:52.221126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 64/130] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:52.226991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668aeba-154b-4463-9a46-cfb0158b7aec None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:52.227475 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668aeba-154b-4463-9a46-cfb0158b7aec None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:52.227560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668aeba-154b-4463-9a46-cfb0158b7aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:52.246811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a668aeba-154b-4463-9a46-cfb0158b7aec None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:52.248041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a668aeba-154b-4463-9a46-cfb0158b7aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:52.317404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 67/131] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:52.589411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c273e52-7bc5-4a47-8ba1-b91e41775aee None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:52.670028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d051c-80bc-449e-9fa2-8d5fbcf95a70 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:52.670420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d051c-80bc-449e-9fa2-8d5fbcf95a70 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:52.670762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d051c-80bc-449e-9fa2-8d5fbcf95a70 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:52.672164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 68/132] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:26:52.674643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:52.677708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6a4f4-c49c-4c91-9042-a0b0b83bcb7a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:52.677998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6a4f4-c49c-4c91-9042-a0b0b83bcb7a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:52.678279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6a4f4-c49c-4c91-9042-a0b0b83bcb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:52.691664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c273e52-7bc5-4a47-8ba1-b91e41775aee None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:52.696413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9bc6a4f4-c49c-4c91-9042-a0b0b83bcb7a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:52.697653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9bc6a4f4-c49c-4c91-9042-a0b0b83bcb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:52.725653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e458c-fb3a-462f-bb55-4ef94ba547d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:52.726309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e458c-fb3a-462f-bb55-4ef94ba547d8 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:52.726593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e458c-fb3a-462f-bb55-4ef94ba547d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:52.726864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e458c-fb3a-462f-bb55-4ef94ba547d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:52.727969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e458c-fb3a-462f-bb55-4ef94ba547d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:52.729455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e458c-fb3a-462f-bb55-4ef94ba547d8 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:52.731025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 65/133] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:52.736635 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e458c-fb3a-462f-bb55-4ef94ba547d8 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:52.759581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:52.768891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 69/134] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:26:52.835643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e458c-fb3a-462f-bb55-4ef94ba547d8 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:52.868141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d76df-278a-41a0-a322-bd7b2c271840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:52.868748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d76df-278a-41a0-a322-bd7b2c271840 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:52.869063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d76df-278a-41a0-a322-bd7b2c271840 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:52.869341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d76df-278a-41a0-a322-bd7b2c271840 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:52.869987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d76df-278a-41a0-a322-bd7b2c271840 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:52.871448 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d76df-278a-41a0-a322-bd7b2c271840 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:52.895154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 66/135] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:26:52 2022] POST /identity/v3/users => generated 283 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:26:53.042448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6a4f4-c49c-4c91-9042-a0b0b83bcb7a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:53.146182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6a4f4-c49c-4c91-9042-a0b0b83bcb7a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:53.189484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bbe48-5137-42ea-95c2-836c0666dd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:53.190762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bbe48-5137-42ea-95c2-836c0666dd6f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:53.190762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bbe48-5137-42ea-95c2-836c0666dd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:53.190762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bbe48-5137-42ea-95c2-836c0666dd6f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:53.201923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bbe48-5137-42ea-95c2-836c0666dd6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-7c2bbe48-5137-42ea-95c2-836c0666dd6f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:53.206842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:53.207689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:26:53.208435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 70/136] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:53.215065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d76df-278a-41a0-a322-bd7b2c271840 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:53.229417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:53.332206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d76df-278a-41a0-a322-bd7b2c271840 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:53.369095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf16cc-e561-4d6d-bb79-68adc4fa3b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:53.369740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf16cc-e561-4d6d-bb79-68adc4fa3b2f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:53.370126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf16cc-e561-4d6d-bb79-68adc4fa3b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:53.370464 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf16cc-e561-4d6d-bb79-68adc4fa3b2f None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:53.371073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf16cc-e561-4d6d-bb79-68adc4fa3b2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:53.371382 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf16cc-e561-4d6d-bb79-68adc4fa3b2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:53.372784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf16cc-e561-4d6d-bb79-68adc4fa3b2f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:53.382614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 67/137] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:26:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:53.387900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bbe48-5137-42ea-95c2-836c0666dd6f None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:53.456248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:53.501339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bbe48-5137-42ea-95c2-836c0666dd6f None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:53.536973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110d812-3131-4d2d-bb7f-3c9c952737f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:53.537632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110d812-3131-4d2d-bb7f-3c9c952737f1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:53.537978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110d812-3131-4d2d-bb7f-3c9c952737f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:53.538264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110d812-3131-4d2d-bb7f-3c9c952737f1 None admin] PATH_INFO: `/v3/users/86762b83e1d04cbfbc95d8f083beca07` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:53.539445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110d812-3131-4d2d-bb7f-3c9c952737f1 None admin] RBAC: Authorizing `identity:get_user(user_id=86762b83e1d04cbfbc95d8f083beca07)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:53.540965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110d812-3131-4d2d-bb7f-3c9c952737f1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:53.542590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 71/138] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:53 2022] GET /identity/v3/users/86762b83e1d04cbfbc95d8f083beca07 => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:26:53.548481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf16cc-e561-4d6d-bb79-68adc4fa3b2f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:53.627047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:53.660859 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf16cc-e561-4d6d-bb79-68adc4fa3b2f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:53.696897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dff618-7d71-4f34-8fbb-92df63d3f84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:53.697532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dff618-7d71-4f34-8fbb-92df63d3f84f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:53.697823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dff618-7d71-4f34-8fbb-92df63d3f84f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:53.698100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dff618-7d71-4f34-8fbb-92df63d3f84f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:53.776981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dff618-7d71-4f34-8fbb-92df63d3f84f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:53.778379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dff618-7d71-4f34-8fbb-92df63d3f84f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:53.849712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:53.883109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 68/139] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:26:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:26:53.889105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110d812-3131-4d2d-bb7f-3c9c952737f1 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:53.996605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110d812-3131-4d2d-bb7f-3c9c952737f1 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:54.029480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:54.032622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a87fb9-d700-4515-abb3-04264e8eda40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:54.033467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a87fb9-d700-4515-abb3-04264e8eda40 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:54.033747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a87fb9-d700-4515-abb3-04264e8eda40 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:54.034032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a87fb9-d700-4515-abb3-04264e8eda40 None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:54.035252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a87fb9-d700-4515-abb3-04264e8eda40 None admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:54.037022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a87fb9-d700-4515-abb3-04264e8eda40 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:54.039150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 72/140] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:53 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:54.045275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dff618-7d71-4f34-8fbb-92df63d3f84f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:54.153304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dff618-7d71-4f34-8fbb-92df63d3f84f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:54.202658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32fdbe-915a-4fc4-85be-41f01bb2ad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:54.203249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fdbe-915a-4fc4-85be-41f01bb2ad7b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:54.203556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fdbe-915a-4fc4-85be-41f01bb2ad7b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:54.203813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fdbe-915a-4fc4-85be-41f01bb2ad7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:54.204506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fdbe-915a-4fc4-85be-41f01bb2ad7b None admin] RBAC: Adding query filter params (role.id=235abd0f332d45e69a86877b4fb95973, scope.project.id=ba4a0bd34a554747b58ff7dc25238f38, user.id=86762b83e1d04cbfbc95d8f083beca07) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:54.204816 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fdbe-915a-4fc4-85be-41f01bb2ad7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:54.206278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fdbe-915a-4fc4-85be-41f01bb2ad7b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:54.212384 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 69/141] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:26:54 2022] GET /identity/v3/role_assignments?user.id=86762b83e1d04cbfbc95d8f083beca07&scope.project.id=ba4a0bd34a554747b58ff7dc25238f38&role.id=235abd0f332d45e69a86877b4fb95973 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:55.286907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605784e-b600-423e-b7ec-599d9fc0c5ed None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:55.287444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605784e-b600-423e-b7ec-599d9fc0c5ed None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:55.287444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605784e-b600-423e-b7ec-599d9fc0c5ed None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:55.288823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 73/142] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:55.298205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a47cc7-20af-4866-9b4b-ceb69f95019c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:55.298689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a47cc7-20af-4866-9b4b-ceb69f95019c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:55.298770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a47cc7-20af-4866-9b4b-ceb69f95019c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:55.317093 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-38a47cc7-20af-4866-9b4b-ceb69f95019c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:55.318277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-38a47cc7-20af-4866-9b4b-ceb69f95019c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:55.346581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ae759-385f-4269-9014-d46595c24ffd None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:55.346921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ae759-385f-4269-9014-d46595c24ffd None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:55.347141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ae759-385f-4269-9014-d46595c24ffd None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:55.349399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 74/143] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:26:55.350174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:55.355479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2e979-490d-4385-bc37-955ee5027a34 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:55.355783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2e979-490d-4385-bc37-955ee5027a34 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:55.356026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2e979-490d-4385-bc37-955ee5027a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:55.374890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b5e2e979-490d-4385-bc37-955ee5027a34 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:55.376097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b5e2e979-490d-4385-bc37-955ee5027a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:55.395429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 70/144] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:55.448465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 75/145] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:26:55.461322 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f7bb0-9a09-4595-89a4-270c08000183 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:55.461681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f7bb0-9a09-4595-89a4-270c08000183 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:55.462025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f7bb0-9a09-4595-89a4-270c08000183 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:55.466740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 71/146] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:55.469266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cf524-897a-4eeb-b5d5-4f4897668f7a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:55.469534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cf524-897a-4eeb-b5d5-4f4897668f7a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:55.469806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cf524-897a-4eeb-b5d5-4f4897668f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:55.488876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-c63cf524-897a-4eeb-b5d5-4f4897668f7a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:55.490330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c63cf524-897a-4eeb-b5d5-4f4897668f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:55.559863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 76/147] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:55.663830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f7bb0-9a09-4595-89a4-270c08000183 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:55.712055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cf524-897a-4eeb-b5d5-4f4897668f7a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:55.769463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f7bb0-9a09-4595-89a4-270c08000183 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:55.771685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:55.812246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4103e5-9bea-490d-94dc-c334db77e583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:55.812925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4103e5-9bea-490d-94dc-c334db77e583 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:55.813225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4103e5-9bea-490d-94dc-c334db77e583 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:55.813496 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4103e5-9bea-490d-94dc-c334db77e583 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:55.824260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4103e5-9bea-490d-94dc-c334db77e583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:55.825367 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cf524-897a-4eeb-b5d5-4f4897668f7a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-ab4103e5-9bea-490d-94dc-c334db77e583 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:55.829771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:55.830641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:55.831592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 72/148] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:26:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:55.836461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4103e5-9bea-490d-94dc-c334db77e583 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:55.862137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:55.867951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d49f6-40fb-4c5d-90b1-405a2d430fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:55.868594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d49f6-40fb-4c5d-90b1-405a2d430fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:55.868894 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d49f6-40fb-4c5d-90b1-405a2d430fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:55.869170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d49f6-40fb-4c5d-90b1-405a2d430fa1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:55.879922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d49f6-40fb-4c5d-90b1-405a2d430fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-846d49f6-40fb-4c5d-90b1-405a2d430fa1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:55.884827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:55.885842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 77/149] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:26:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:26:55.892134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d49f6-40fb-4c5d-90b1-405a2d430fa1 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:55.936391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4103e5-9bea-490d-94dc-c334db77e583 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:55.970984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdaf597-5290-4ad0-a7e4-a06d5c86e328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:55.971513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdaf597-5290-4ad0-a7e4-a06d5c86e328 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:55.971794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdaf597-5290-4ad0-a7e4-a06d5c86e328 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:55.972066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdaf597-5290-4ad0-a7e4-a06d5c86e328 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:55.972732 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdaf597-5290-4ad0-a7e4-a06d5c86e328 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:55.973069 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdaf597-5290-4ad0-a7e4-a06d5c86e328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:55.974457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdaf597-5290-4ad0-a7e4-a06d5c86e328 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:55.982558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 73/150] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:26:55.984467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d49f6-40fb-4c5d-90b1-405a2d430fa1 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:55.985796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdaf597-5290-4ad0-a7e4-a06d5c86e328 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:56.001165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:56.022357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f44c6-4d76-4c9f-8719-7193b278d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:56.022996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f44c6-4d76-4c9f-8719-7193b278d29d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:56.023279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f44c6-4d76-4c9f-8719-7193b278d29d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:56.023551 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f44c6-4d76-4c9f-8719-7193b278d29d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:56.025382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f44c6-4d76-4c9f-8719-7193b278d29d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:56.026835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f44c6-4d76-4c9f-8719-7193b278d29d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:56.028582 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 78/151] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:56.029229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f44c6-4d76-4c9f-8719-7193b278d29d None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:56.073396 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:56.087763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdaf597-5290-4ad0-a7e4-a06d5c86e328 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:56.123441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82fe04-a739-41a4-ad32-9baeb5c2e15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:56.124094 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82fe04-a739-41a4-ad32-9baeb5c2e15b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:56.125092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82fe04-a739-41a4-ad32-9baeb5c2e15b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:56.125374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82fe04-a739-41a4-ad32-9baeb5c2e15b None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:56.126048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82fe04-a739-41a4-ad32-9baeb5c2e15b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:56.126354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82fe04-a739-41a4-ad32-9baeb5c2e15b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:56.127759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82fe04-a739-41a4-ad32-9baeb5c2e15b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:56.130965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f44c6-4d76-4c9f-8719-7193b278d29d None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:56.136721 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 74/152] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:26:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:56.136721 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82fe04-a739-41a4-ad32-9baeb5c2e15b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:56.171770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d7b81-9c33-4364-8670-aba62345b11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:56.172418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d7b81-9c33-4364-8670-aba62345b11c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:56.172693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d7b81-9c33-4364-8670-aba62345b11c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:56.172991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d7b81-9c33-4364-8670-aba62345b11c None admin] PATH_INFO: `/v3/users/b55d059a28f040e39423ffc7719dc90d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:56.189026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d7b81-9c33-4364-8670-aba62345b11c None admin] RBAC: Authorizing `identity:get_user(user_id=b55d059a28f040e39423ffc7719dc90d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:56.190562 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d7b81-9c33-4364-8670-aba62345b11c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:56.192293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 79/153] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:56 2022] GET /identity/v3/users/b55d059a28f040e39423ffc7719dc90d => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:56.193136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d7b81-9c33-4364-8670-aba62345b11c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:56.239382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:56.248399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82fe04-a739-41a4-ad32-9baeb5c2e15b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:56.286114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b89e4-a04f-4b46-91d5-2bb22667540a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:56.286925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b89e4-a04f-4b46-91d5-2bb22667540a None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:56.287023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b89e4-a04f-4b46-91d5-2bb22667540a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:56.287260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b89e4-a04f-4b46-91d5-2bb22667540a None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:56.288376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b89e4-a04f-4b46-91d5-2bb22667540a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:56.289466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b89e4-a04f-4b46-91d5-2bb22667540a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:56.301142 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d7b81-9c33-4364-8670-aba62345b11c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:56.317776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 75/154] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:26:56 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:56.318625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b89e4-a04f-4b46-91d5-2bb22667540a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:56.322247 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:56.344053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec6f97-48dd-44fb-8d8f-2fd99d5f4c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:56.344706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec6f97-48dd-44fb-8d8f-2fd99d5f4c2c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:56.345028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec6f97-48dd-44fb-8d8f-2fd99d5f4c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:56.345352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec6f97-48dd-44fb-8d8f-2fd99d5f4c2c None admin] PATH_INFO: `/v3/users/b55d059a28f040e39423ffc7719dc90d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:56.346619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec6f97-48dd-44fb-8d8f-2fd99d5f4c2c None admin] RBAC: Authorizing `identity:get_user(user_id=b55d059a28f040e39423ffc7719dc90d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:56.348143 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec6f97-48dd-44fb-8d8f-2fd99d5f4c2c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:56.350095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 80/155] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:56 2022] GET /identity/v3/users/b55d059a28f040e39423ffc7719dc90d => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:56.355883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec6f97-48dd-44fb-8d8f-2fd99d5f4c2c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:56.468697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b89e4-a04f-4b46-91d5-2bb22667540a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:56.492863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec6f97-48dd-44fb-8d8f-2fd99d5f4c2c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:56.503265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:56.526119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5b6e5-cdad-44e5-8cdd-18c619ea0474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:56.527088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5b6e5-cdad-44e5-8cdd-18c619ea0474 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:56.527532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5b6e5-cdad-44e5-8cdd-18c619ea0474 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:56.527961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5b6e5-cdad-44e5-8cdd-18c619ea0474 None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:56.529872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5b6e5-cdad-44e5-8cdd-18c619ea0474 None admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:56.532967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5b6e5-cdad-44e5-8cdd-18c619ea0474 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:56.533828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c4e66-6368-418b-a3e6-d0fb990fc8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:56.534462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c4e66-6368-418b-a3e6-d0fb990fc8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:56.534736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c4e66-6368-418b-a3e6-d0fb990fc8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:56.535006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c4e66-6368-418b-a3e6-d0fb990fc8a2 None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:56.535696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 76/156] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:56 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:56.536250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c4e66-6368-418b-a3e6-d0fb990fc8a2 None admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:56.537800 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c4e66-6368-418b-a3e6-d0fb990fc8a2 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:56.539801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 81/157] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:56 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:26:56.541517 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5b6e5-cdad-44e5-8cdd-18c619ea0474 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:56.545471 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c4e66-6368-418b-a3e6-d0fb990fc8a2 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:56.628673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:56.674197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c4e66-6368-418b-a3e6-d0fb990fc8a2 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:56.676511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5b6e5-cdad-44e5-8cdd-18c619ea0474 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:56.721571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c43398-3007-4ab7-966d-67dd482917eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:56.722240 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43398-3007-4ab7-966d-67dd482917eb None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:56.722514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43398-3007-4ab7-966d-67dd482917eb None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:56.722783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43398-3007-4ab7-966d-67dd482917eb None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/users/b55d059a28f040e39423ffc7719dc90d/roles/04b8e62238cb4198928fd327ef78e6ae` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:56.726006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c43398-3007-4ab7-966d-67dd482917eb None admin] RBAC: Authorizing `identity:create_grant(project_id=1073f38ce4ec449ba0477f2e9e1786ce, user_id=b55d059a28f040e39423ffc7719dc90d, role_id=04b8e62238cb4198928fd327ef78e6ae)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:56.726963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ff157-3dae-45a8-9e8e-49230b5c947f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:56.727514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c43398-3007-4ab7-966d-67dd482917eb None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:56.727683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ff157-3dae-45a8-9e8e-49230b5c947f None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:56.727971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ff157-3dae-45a8-9e8e-49230b5c947f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:56.728247 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ff157-3dae-45a8-9e8e-49230b5c947f None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/users/b55d059a28f040e39423ffc7719dc90d/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:56.730383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ff157-3dae-45a8-9e8e-49230b5c947f None admin] RBAC: Authorizing `identity:create_grant(project_id=1073f38ce4ec449ba0477f2e9e1786ce, user_id=b55d059a28f040e39423ffc7719dc90d, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:56.731988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ff157-3dae-45a8-9e8e-49230b5c947f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:56.745824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 82/158] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:26:56 2022] PUT /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/users/b55d059a28f040e39423ffc7719dc90d/roles/04b8e62238cb4198928fd327ef78e6ae => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:26:56.747260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 77/159] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:26:56 2022] PUT /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/users/b55d059a28f040e39423ffc7719dc90d/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:26:58.644450 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e240ca-efdd-44f7-ae87-056359572dfb None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:58.645016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e240ca-efdd-44f7-ae87-056359572dfb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:58.645016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e240ca-efdd-44f7-ae87-056359572dfb None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:58.646501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 83/160] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:58.652056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bee92-d80c-4bc1-b979-5f1619a55575 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:58.652479 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bee92-d80c-4bc1-b979-5f1619a55575 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:58.652677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bee92-d80c-4bc1-b979-5f1619a55575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:58.671717 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1e9bee92-d80c-4bc1-b979-5f1619a55575 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:58.672915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1e9bee92-d80c-4bc1-b979-5f1619a55575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:58.725912 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f73a95-8021-4de6-b6f8-482aa209b258 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:58.726212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f73a95-8021-4de6-b6f8-482aa209b258 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:58.726460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f73a95-8021-4de6-b6f8-482aa209b258 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:58.727956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 84/161] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:26:58.732947 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4bad2-f9ed-45c2-bca0-0e2d8c2f1552 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:58.733235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4bad2-f9ed-45c2-bca0-0e2d8c2f1552 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:58.733546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4bad2-f9ed-45c2-bca0-0e2d8c2f1552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:58.744051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 78/162] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:58.752555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a7c4bad2-f9ed-45c2-bca0-0e2d8c2f1552 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:58.753796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a7c4bad2-f9ed-45c2-bca0-0e2d8c2f1552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:58.783934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:58.839945 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 85/163] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:26:58.994167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea6f8a-6bdf-49a3-b747-dd7b2c2de676 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:58.994833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea6f8a-6bdf-49a3-b747-dd7b2c2de676 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:58.995338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea6f8a-6bdf-49a3-b747-dd7b2c2de676 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:58.996929 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 79/164] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:59.003116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266edb39-1b96-4389-a93e-bf46629b2c05 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.003664 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266edb39-1b96-4389-a93e-bf46629b2c05 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.004169 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266edb39-1b96-4389-a93e-bf46629b2c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.023398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-266edb39-1b96-4389-a93e-bf46629b2c05 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:59.024951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-266edb39-1b96-4389-a93e-bf46629b2c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:59.028049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea6f8a-6bdf-49a3-b747-dd7b2c2de676 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:59.100977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 86/165] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:26:59.100977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102ddb-8ed5-4114-85d6-ce22090c7094 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.100977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102ddb-8ed5-4114-85d6-ce22090c7094 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.100977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102ddb-8ed5-4114-85d6-ce22090c7094 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.100977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 87/166] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:26:59.110468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:59.110633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd969305-f3fb-4920-8232-8d63016dcd61 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.111084 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd969305-f3fb-4920-8232-8d63016dcd61 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.111417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd969305-f3fb-4920-8232-8d63016dcd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.130613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-fd969305-f3fb-4920-8232-8d63016dcd61 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:26:59.131808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fd969305-f3fb-4920-8232-8d63016dcd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:26:59.164093 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea6f8a-6bdf-49a3-b747-dd7b2c2de676 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:59.219613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30082a6-65d1-4bab-afe1-1c55a59194be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:59.220530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30082a6-65d1-4bab-afe1-1c55a59194be None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.220807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30082a6-65d1-4bab-afe1-1c55a59194be None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.221109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30082a6-65d1-4bab-afe1-1c55a59194be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.221683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 88/167] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:26:59.222345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd969305-f3fb-4920-8232-8d63016dcd61 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:59.236495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30082a6-65d1-4bab-afe1-1c55a59194be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-b30082a6-65d1-4bab-afe1-1c55a59194be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:59.239881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:59.240748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:59.242132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 80/168] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:26:59.246775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30082a6-65d1-4bab-afe1-1c55a59194be None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:59.267910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:59.364043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd969305-f3fb-4920-8232-8d63016dcd61 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:59.377099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30082a6-65d1-4bab-afe1-1c55a59194be None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:59.405379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:59.409752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25e0c-0ea3-4b08-909e-63dd49a9297f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:59.410478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25e0c-0ea3-4b08-909e-63dd49a9297f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.410761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25e0c-0ea3-4b08-909e-63dd49a9297f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.411047 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25e0c-0ea3-4b08-909e-63dd49a9297f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.412150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d25e0c-0ea3-4b08-909e-63dd49a9297f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:59.413659 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d25e0c-0ea3-4b08-909e-63dd49a9297f None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:59.415380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 89/169] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:26:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:59.415999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25e0c-0ea3-4b08-909e-63dd49a9297f None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:59.427202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32d53e-f629-401a-a54e-28bebb6ceb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:59.428225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32d53e-f629-401a-a54e-28bebb6ceb5b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.428703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32d53e-f629-401a-a54e-28bebb6ceb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.429411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32d53e-f629-401a-a54e-28bebb6ceb5b None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.430328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32d53e-f629-401a-a54e-28bebb6ceb5b None admin] RBAC: Adding query filter params (name=member) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:59.430849 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32d53e-f629-401a-a54e-28bebb6ceb5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:59.432576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32d53e-f629-401a-a54e-28bebb6ceb5b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:59.444782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 81/170] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:26:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:26:59.446438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32d53e-f629-401a-a54e-28bebb6ceb5b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:59.523608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:59.531269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25e0c-0ea3-4b08-909e-63dd49a9297f None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:59.558478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32d53e-f629-401a-a54e-28bebb6ceb5b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:59.566479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bc39b-aa55-4785-ac63-a06b02a68e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:59.567116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bc39b-aa55-4785-ac63-a06b02a68e4b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.567394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bc39b-aa55-4785-ac63-a06b02a68e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.567661 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bc39b-aa55-4785-ac63-a06b02a68e4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.577728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bc39b-aa55-4785-ac63-a06b02a68e4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-ba7bc39b-aa55-4785-ac63-a06b02a68e4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:59.583023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:59.583887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:26:59.584667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 90/171] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:26:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:59.585384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bc39b-aa55-4785-ac63-a06b02a68e4b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:59.591745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27a16a-a435-4989-a5ab-2084cb8fef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:59.592359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27a16a-a435-4989-a5ab-2084cb8fef75 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.592635 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27a16a-a435-4989-a5ab-2084cb8fef75 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.593484 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27a16a-a435-4989-a5ab-2084cb8fef75 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.594144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27a16a-a435-4989-a5ab-2084cb8fef75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:59.595611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27a16a-a435-4989-a5ab-2084cb8fef75 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:59.623871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 82/172] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:26:59 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:26:59.623871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27a16a-a435-4989-a5ab-2084cb8fef75 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:59.675140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:59.716415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bc39b-aa55-4785-ac63-a06b02a68e4b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:59.751163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27a16a-a435-4989-a5ab-2084cb8fef75 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:59.754370 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae31b5a-1150-43fb-a36e-cbb105f121c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:59.755037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae31b5a-1150-43fb-a36e-cbb105f121c8 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.755316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae31b5a-1150-43fb-a36e-cbb105f121c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.755589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae31b5a-1150-43fb-a36e-cbb105f121c8 None admin] PATH_INFO: `/v3/users/4c64e6fbe1d44e4d9368dce8bd8c5f5e` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.770665 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae31b5a-1150-43fb-a36e-cbb105f121c8 None admin] RBAC: Authorizing `identity:get_user(user_id=4c64e6fbe1d44e4d9368dce8bd8c5f5e)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:59.772208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae31b5a-1150-43fb-a36e-cbb105f121c8 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:59.776963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 91/173] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:26:59 2022] GET /identity/v3/users/4c64e6fbe1d44e4d9368dce8bd8c5f5e => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:59.776963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae31b5a-1150-43fb-a36e-cbb105f121c8 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:59.793750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:59.800346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5b4ae-7f99-4c55-b791-7ded8a59632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:59.801025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5b4ae-7f99-4c55-b791-7ded8a59632b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.801309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5b4ae-7f99-4c55-b791-7ded8a59632b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.801580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5b4ae-7f99-4c55-b791-7ded8a59632b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.815151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5b4ae-7f99-4c55-b791-7ded8a59632b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-96f5b4ae-7f99-4c55-b791-7ded8a59632b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:26:59.820778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:26:59.821915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:26:59.822676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 83/174] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:26:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:26:59.823313 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5b4ae-7f99-4c55-b791-7ded8a59632b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:26:59.915607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae31b5a-1150-43fb-a36e-cbb105f121c8 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:59.942596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:26:59.946256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5b4ae-7f99-4c55-b791-7ded8a59632b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:26:59.957577 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e1f2d-232d-4512-901d-da361348e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:26:59.958239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e1f2d-232d-4512-901d-da361348e13a None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:26:59.958527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e1f2d-232d-4512-901d-da361348e13a None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:26:59.958945 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e1f2d-232d-4512-901d-da361348e13a None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:26:59.959603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e1f2d-232d-4512-901d-da361348e13a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:26:59.959942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e1f2d-232d-4512-901d-da361348e13a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:26:59.961546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e1f2d-232d-4512-901d-da361348e13a None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:26:59.973465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 92/175] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:26:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:26:59.974709 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e1f2d-232d-4512-901d-da361348e13a None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:00.004266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8abc5-8650-4b83-91b9-86bce6194edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:00.004979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8abc5-8650-4b83-91b9-86bce6194edc None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:00.005552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8abc5-8650-4b83-91b9-86bce6194edc None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:00.005835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8abc5-8650-4b83-91b9-86bce6194edc None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:00.007180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8abc5-8650-4b83-91b9-86bce6194edc None admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:00.009053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8abc5-8650-4b83-91b9-86bce6194edc None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:00.011243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 84/176] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:26:59 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:00.012027 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8abc5-8650-4b83-91b9-86bce6194edc None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:00.075375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:00.107797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e1f2d-232d-4512-901d-da361348e13a None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:00.145927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8abc5-8650-4b83-91b9-86bce6194edc None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:00.147212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32b23b-e1c0-4a0b-b974-366b055dfc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:00.147893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32b23b-e1c0-4a0b-b974-366b055dfc1a None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:00.148176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32b23b-e1c0-4a0b-b974-366b055dfc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:00.148603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32b23b-e1c0-4a0b-b974-366b055dfc1a None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:00.149263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32b23b-e1c0-4a0b-b974-366b055dfc1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:00.149574 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32b23b-e1c0-4a0b-b974-366b055dfc1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:00.151067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32b23b-e1c0-4a0b-b974-366b055dfc1a None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:00.165252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 93/177] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:26:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:00.165252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32b23b-e1c0-4a0b-b974-366b055dfc1a None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:00.202981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5898b-15fe-4297-87a9-146df447386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:00.203630 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5898b-15fe-4297-87a9-146df447386d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:00.203912 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5898b-15fe-4297-87a9-146df447386d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:00.204186 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5898b-15fe-4297-87a9-146df447386d None admin] PATH_INFO: `/v3/users/b55d059a28f040e39423ffc7719dc90d` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:00.205526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5898b-15fe-4297-87a9-146df447386d None admin] RBAC: Authorizing `identity:get_user(user_id=b55d059a28f040e39423ffc7719dc90d)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:00.206996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5898b-15fe-4297-87a9-146df447386d None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:00.208763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 85/178] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:00 2022] GET /identity/v3/users/b55d059a28f040e39423ffc7719dc90d => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:00.214323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5898b-15fe-4297-87a9-146df447386d None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:00.265356 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:00.292118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32b23b-e1c0-4a0b-b974-366b055dfc1a None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:00.329581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da6f2-d869-41f2-af6e-b31225923897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:00.330294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da6f2-d869-41f2-af6e-b31225923897 None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:00.330673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da6f2-d869-41f2-af6e-b31225923897 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:00.331018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da6f2-d869-41f2-af6e-b31225923897 None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/users/4c64e6fbe1d44e4d9368dce8bd8c5f5e/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:00.333395 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459da6f2-d869-41f2-af6e-b31225923897 None admin] RBAC: Authorizing `identity:create_grant(project_id=1073f38ce4ec449ba0477f2e9e1786ce, user_id=4c64e6fbe1d44e4d9368dce8bd8c5f5e, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:00.335694 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459da6f2-d869-41f2-af6e-b31225923897 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:00.338909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:00.352613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5898b-15fe-4297-87a9-146df447386d None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:00.353552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 94/179] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:27:00 2022] PUT /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/users/4c64e6fbe1d44e4d9368dce8bd8c5f5e/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:00.354684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da6f2-d869-41f2-af6e-b31225923897 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:00.427027 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff1ca0-1712-4f8d-b14f-0af20efe0d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:00.428095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1ca0-1712-4f8d-b14f-0af20efe0d7b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:00.428559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1ca0-1712-4f8d-b14f-0af20efe0d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:00.429139 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff1ca0-1712-4f8d-b14f-0af20efe0d7b None admin] PATH_INFO: `/v3/users/b55d059a28f040e39423ffc7719dc90d` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:00.431508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff1ca0-1712-4f8d-b14f-0af20efe0d7b None admin] RBAC: Authorizing `identity:get_user(user_id=b55d059a28f040e39423ffc7719dc90d)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:00.433509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff1ca0-1712-4f8d-b14f-0af20efe0d7b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:00.435499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 86/180] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:00 2022] GET /identity/v3/users/b55d059a28f040e39423ffc7719dc90d => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:00.442717 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff1ca0-1712-4f8d-b14f-0af20efe0d7b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:00.574667 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff1ca0-1712-4f8d-b14f-0af20efe0d7b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:00.579954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:00.613645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da6f2-d869-41f2-af6e-b31225923897 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:00.630388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2eac9b-70d7-4b25-8145-813e91d46061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:00.631550 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2eac9b-70d7-4b25-8145-813e91d46061 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:00.632015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2eac9b-70d7-4b25-8145-813e91d46061 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:00.632615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2eac9b-70d7-4b25-8145-813e91d46061 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:00.655536 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ba51b-772e-416f-8250-d895c352992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:00.656260 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ba51b-772e-416f-8250-d895c352992b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:00.656610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ba51b-772e-416f-8250-d895c352992b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:00.657291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ba51b-772e-416f-8250-d895c352992b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:00.724129 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2eac9b-70d7-4b25-8145-813e91d46061 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:00.726460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2eac9b-70d7-4b25-8145-813e91d46061 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:00.736037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:00.759977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ba51b-772e-416f-8250-d895c352992b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:00.763738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ba51b-772e-416f-8250-d895c352992b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:00.938525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 87/181] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 18 11:27:00.944072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2eac9b-70d7-4b25-8145-813e91d46061 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:00.947012 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:00.985604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 95/182] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:27:00.991191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ba51b-772e-416f-8250-d895c352992b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:01.106630 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2eac9b-70d7-4b25-8145-813e91d46061 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:01.142982 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c335cd-ca65-4162-9244-90eeefb56072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:01.143634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c335cd-ca65-4162-9244-90eeefb56072 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:01.143932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c335cd-ca65-4162-9244-90eeefb56072 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:01.144181 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c335cd-ca65-4162-9244-90eeefb56072 None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:01.145521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c335cd-ca65-4162-9244-90eeefb56072 None admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:01.146979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c335cd-ca65-4162-9244-90eeefb56072 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:01.148898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 88/183] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:27:00 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:01.157902 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c335cd-ca65-4162-9244-90eeefb56072 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:01.185831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:01.194961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ba51b-772e-416f-8250-d895c352992b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:01.269839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e056ffe-48b5-4ba9-ae3f-7b9dfa527e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:01.270498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056ffe-48b5-4ba9-ae3f-7b9dfa527e63 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:01.270772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056ffe-48b5-4ba9-ae3f-7b9dfa527e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:01.271043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056ffe-48b5-4ba9-ae3f-7b9dfa527e63 None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:01.276025 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e056ffe-48b5-4ba9-ae3f-7b9dfa527e63 None admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:01.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e056ffe-48b5-4ba9-ae3f-7b9dfa527e63 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:01.281876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 96/184] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:27:00 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:01.287561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e056ffe-48b5-4ba9-ae3f-7b9dfa527e63 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:01.315554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:01.376049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c335cd-ca65-4162-9244-90eeefb56072 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:01.437579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065e126-ea10-40c9-af87-773dae02b43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:01.438187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065e126-ea10-40c9-af87-773dae02b43c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:01.438461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065e126-ea10-40c9-af87-773dae02b43c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:01.438728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065e126-ea10-40c9-af87-773dae02b43c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:01.439626 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065e126-ea10-40c9-af87-773dae02b43c None admin] RBAC: Adding query filter params (role.id=8d9ff9da6ad946a6bc49336777182ff9, scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce, user.id=b55d059a28f040e39423ffc7719dc90d) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:01.439735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065e126-ea10-40c9-af87-773dae02b43c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:01.444865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065e126-ea10-40c9-af87-773dae02b43c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:01.447008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 89/185] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:01 2022] GET /identity/v3/role_assignments?user.id=b55d059a28f040e39423ffc7719dc90d&scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce&role.id=8d9ff9da6ad946a6bc49336777182ff9 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:01.476890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e056ffe-48b5-4ba9-ae3f-7b9dfa527e63 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:01.536624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4c1d4-09c4-4566-8fe4-3eada2448969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:01.537644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4c1d4-09c4-4566-8fe4-3eada2448969 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:01.538162 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4c1d4-09c4-4566-8fe4-3eada2448969 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:01.538637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4c1d4-09c4-4566-8fe4-3eada2448969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:01.539596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4c1d4-09c4-4566-8fe4-3eada2448969 None admin] RBAC: Adding query filter params (role.id=04b8e62238cb4198928fd327ef78e6ae, scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce, user.id=b55d059a28f040e39423ffc7719dc90d) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:01.540196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4c1d4-09c4-4566-8fe4-3eada2448969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:01.541955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4c1d4-09c4-4566-8fe4-3eada2448969 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:01.549556 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 97/186] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:01 2022] GET /identity/v3/role_assignments?user.id=b55d059a28f040e39423ffc7719dc90d&scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce&role.id=04b8e62238cb4198928fd327ef78e6ae => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:02.176290 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b48e30-3291-47aa-af78-1e0f9b8250c4 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:02.176800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b48e30-3291-47aa-af78-1e0f9b8250c4 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:02.176901 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b48e30-3291-47aa-af78-1e0f9b8250c4 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:02.178198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 90/187] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:02.183788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b243b90-be80-452f-8419-7795ba6ede2b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:02.184204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b243b90-be80-452f-8419-7795ba6ede2b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:02.184368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b243b90-be80-452f-8419-7795ba6ede2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:02.203293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-5b243b90-be80-452f-8419-7795ba6ede2b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:02.204472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5b243b90-be80-452f-8419-7795ba6ede2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:02.225266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68878-8dd6-47c9-b604-3b4f19cc7321 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:02.225568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68878-8dd6-47c9-b604-3b4f19cc7321 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:02.225819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68878-8dd6-47c9-b604-3b4f19cc7321 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:02.227151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 91/188] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:27:02.232152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02db4a5-44ed-4776-9c7e-70a9f584dc81 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:02.232827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02db4a5-44ed-4776-9c7e-70a9f584dc81 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:02.232827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02db4a5-44ed-4776-9c7e-70a9f584dc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:02.235925 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:02.251073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e02db4a5-44ed-4776-9c7e-70a9f584dc81 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:02.252253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e02db4a5-44ed-4776-9c7e-70a9f584dc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:02.289996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 98/189] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:02.323321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 92/190] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:27:02.554417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b243b90-be80-452f-8419-7795ba6ede2b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:02.587977 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02db4a5-44ed-4776-9c7e-70a9f584dc81 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:02.657613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:02.661987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b243b90-be80-452f-8419-7795ba6ede2b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:02.700035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02db4a5-44ed-4776-9c7e-70a9f584dc81 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:02.701066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc32579-db35-474a-8c3d-18608e9fbfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:02.701707 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc32579-db35-474a-8c3d-18608e9fbfa0 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:02.701981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc32579-db35-474a-8c3d-18608e9fbfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:02.702283 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc32579-db35-474a-8c3d-18608e9fbfa0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:02.714885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc32579-db35-474a-8c3d-18608e9fbfa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-dbc32579-db35-474a-8c3d-18608e9fbfa0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:02.720519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:02.721673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:02.722596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 99/191] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:02.727776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc32579-db35-474a-8c3d-18608e9fbfa0 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:02.746741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732a51e-756d-49e6-af15-c8a617928e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:02.748430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732a51e-756d-49e6-af15-c8a617928e99 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:02.749172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732a51e-756d-49e6-af15-c8a617928e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:02.749825 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732a51e-756d-49e6-af15-c8a617928e99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:02.752088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732a51e-756d-49e6-af15-c8a617928e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:02.755158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732a51e-756d-49e6-af15-c8a617928e99 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:02.758079 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:02.759771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 93/192] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:02.765627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732a51e-756d-49e6-af15-c8a617928e99 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:02.866487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc32579-db35-474a-8c3d-18608e9fbfa0 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:02.900468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8dd1b-c109-4a9e-9af6-283f60947453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:02.901213 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8dd1b-c109-4a9e-9af6-283f60947453 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:02.901506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8dd1b-c109-4a9e-9af6-283f60947453 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:02.901774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8dd1b-c109-4a9e-9af6-283f60947453 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:02.902437 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8dd1b-c109-4a9e-9af6-283f60947453 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:02.902741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8dd1b-c109-4a9e-9af6-283f60947453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:02.904124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8dd1b-c109-4a9e-9af6-283f60947453 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:02.908144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732a51e-756d-49e6-af15-c8a617928e99 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:02.912755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 100/193] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:02.915050 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92950526-dbe6-47c2-97a1-1caa896cc009 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:02.915334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92950526-dbe6-47c2-97a1-1caa896cc009 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:02.915602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92950526-dbe6-47c2-97a1-1caa896cc009 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:02.917061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 101/194] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:02.918180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92950526-dbe6-47c2-97a1-1caa896cc009 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:02.947116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af7a4e-b4c9-4c79-8db5-d230ed33f1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:02.947730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af7a4e-b4c9-4c79-8db5-d230ed33f1f0 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:02.948008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af7a4e-b4c9-4c79-8db5-d230ed33f1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:02.948280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af7a4e-b4c9-4c79-8db5-d230ed33f1f0 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:02.948953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af7a4e-b4c9-4c79-8db5-d230ed33f1f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:02.950386 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af7a4e-b4c9-4c79-8db5-d230ed33f1f0 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:02.958808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:02.972182 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 94/195] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:27:02 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:27:02.973416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7760-e592-4249-bc0b-9428b7cdd730 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:02.973697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7760-e592-4249-bc0b-9428b7cdd730 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:02.973971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e7760-e592-4249-bc0b-9428b7cdd730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:02.994804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1e0e7760-e592-4249-bc0b-9428b7cdd730 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:02.996035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1e0e7760-e592-4249-bc0b-9428b7cdd730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:03.005623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:03.033414 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92950526-dbe6-47c2-97a1-1caa896cc009 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:03.075761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad224017-5bc5-4900-9017-d5e62bc41136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:03.076418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad224017-5bc5-4900-9017-d5e62bc41136 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:03.076700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad224017-5bc5-4900-9017-d5e62bc41136 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:03.077007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad224017-5bc5-4900-9017-d5e62bc41136 None admin] PATH_INFO: `/v3/users/c6c84e72064c45358de3cd5ba8ff8c7f` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:03.082850 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 95/196] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:03.092395 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad224017-5bc5-4900-9017-d5e62bc41136 None admin] RBAC: Authorizing `identity:get_user(user_id=c6c84e72064c45358de3cd5ba8ff8c7f)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:03.093941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad224017-5bc5-4900-9017-d5e62bc41136 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:03.095895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 102/197] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:02 2022] GET /identity/v3/users/c6c84e72064c45358de3cd5ba8ff8c7f => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:03.102568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e7760-e592-4249-bc0b-9428b7cdd730 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:03.216040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e7760-e592-4249-bc0b-9428b7cdd730 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:03.251112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:03.266324 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dca0d1-ac45-46d3-b2e3-531dc8b41e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:03.266975 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca0d1-ac45-46d3-b2e3-531dc8b41e97 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:03.267254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca0d1-ac45-46d3-b2e3-531dc8b41e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:03.267527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca0d1-ac45-46d3-b2e3-531dc8b41e97 None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:03.268666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dca0d1-ac45-46d3-b2e3-531dc8b41e97 None admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:03.270177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dca0d1-ac45-46d3-b2e3-531dc8b41e97 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:03.271915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 96/198] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:27:03 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:03.277851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad224017-5bc5-4900-9017-d5e62bc41136 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:03.365957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dca0d1-ac45-46d3-b2e3-531dc8b41e97 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:03.398615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad224017-5bc5-4900-9017-d5e62bc41136 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:03.450364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1356bee-bdbf-423b-9c18-99dc58b9ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:03.450974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1356bee-bdbf-423b-9c18-99dc58b9ccf0 None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:03.451248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1356bee-bdbf-423b-9c18-99dc58b9ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:03.451547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1356bee-bdbf-423b-9c18-99dc58b9ccf0 None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/users/c6c84e72064c45358de3cd5ba8ff8c7f/roles/235abd0f332d45e69a86877b4fb95973` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:03.454618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1356bee-bdbf-423b-9c18-99dc58b9ccf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1073f38ce4ec449ba0477f2e9e1786ce, user_id=c6c84e72064c45358de3cd5ba8ff8c7f, role_id=235abd0f332d45e69a86877b4fb95973)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:03.456108 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1356bee-bdbf-423b-9c18-99dc58b9ccf0 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:03.470206 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:03.476645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 103/199] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:27:03 2022] PUT /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/users/c6c84e72064c45358de3cd5ba8ff8c7f/roles/235abd0f332d45e69a86877b4fb95973 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:27:03.512217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dca0d1-ac45-46d3-b2e3-531dc8b41e97 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:03.554539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3d773-97ad-4dd1-ba16-b5ac81ff726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:03.555236 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3d773-97ad-4dd1-ba16-b5ac81ff726f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:03.555519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3d773-97ad-4dd1-ba16-b5ac81ff726f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:03.555793 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3d773-97ad-4dd1-ba16-b5ac81ff726f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:03.568723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3d773-97ad-4dd1-ba16-b5ac81ff726f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-08f3d773-97ad-4dd1-ba16-b5ac81ff726f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:03.573808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:03.574722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:03.575500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 97/200] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:03.581139 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1356bee-bdbf-423b-9c18-99dc58b9ccf0 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:03.699559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1356bee-bdbf-423b-9c18-99dc58b9ccf0 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:03.735176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d9ea0-a369-448c-a0b7-eb7142ece79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:03.735771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d9ea0-a369-448c-a0b7-eb7142ece79c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:03.736045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d9ea0-a369-448c-a0b7-eb7142ece79c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:03.736346 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d9ea0-a369-448c-a0b7-eb7142ece79c None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:03.737053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d9ea0-a369-448c-a0b7-eb7142ece79c None admin] RBAC: Adding query filter params (name=member) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:03.737355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d9ea0-a369-448c-a0b7-eb7142ece79c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:03.739015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d9ea0-a369-448c-a0b7-eb7142ece79c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:03.751882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 104/201] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:03.757612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3d773-97ad-4dd1-ba16-b5ac81ff726f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:03.813515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:03.893509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3d773-97ad-4dd1-ba16-b5ac81ff726f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:03.948878 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70876e95-708c-4191-9d40-8ca4ce21795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:03.950195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70876e95-708c-4191-9d40-8ca4ce21795b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:03.950885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70876e95-708c-4191-9d40-8ca4ce21795b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:03.951659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70876e95-708c-4191-9d40-8ca4ce21795b None admin] PATH_INFO: `/v3/users/4c64e6fbe1d44e4d9368dce8bd8c5f5e` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:03.954229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70876e95-708c-4191-9d40-8ca4ce21795b None admin] RBAC: Authorizing `identity:get_user(user_id=4c64e6fbe1d44e4d9368dce8bd8c5f5e)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:03.956893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70876e95-708c-4191-9d40-8ca4ce21795b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:03.960204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 98/202] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:03 2022] GET /identity/v3/users/4c64e6fbe1d44e4d9368dce8bd8c5f5e => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:03.965998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d9ea0-a369-448c-a0b7-eb7142ece79c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:04.027777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:04.073297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d9ea0-a369-448c-a0b7-eb7142ece79c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:04.110038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435fdfd-edb6-4acf-83e5-24b92472374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:04.110996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435fdfd-edb6-4acf-83e5-24b92472374d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:04.111426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435fdfd-edb6-4acf-83e5-24b92472374d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:04.111861 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435fdfd-edb6-4acf-83e5-24b92472374d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:04.186451 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435fdfd-edb6-4acf-83e5-24b92472374d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:04.187966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435fdfd-edb6-4acf-83e5-24b92472374d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:04.271964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:04.346754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 105/203] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:27:04.353954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70876e95-708c-4191-9d40-8ca4ce21795b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:04.457965 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70876e95-708c-4191-9d40-8ca4ce21795b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:04.476920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:04.497584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24e16c-206d-427e-8e17-e84c5dd46295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:04.498232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24e16c-206d-427e-8e17-e84c5dd46295 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:04.498511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24e16c-206d-427e-8e17-e84c5dd46295 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:04.498782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24e16c-206d-427e-8e17-e84c5dd46295 None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:04.499953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24e16c-206d-427e-8e17-e84c5dd46295 None admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:04.501426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24e16c-206d-427e-8e17-e84c5dd46295 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:04.503224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 99/204] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:27:04 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:04.509090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435fdfd-edb6-4acf-83e5-24b92472374d None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:04.619466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435fdfd-edb6-4acf-83e5-24b92472374d None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:04.656952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf62fd-2ebb-41a4-b2a6-a6e400e29ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:04.657568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf62fd-2ebb-41a4-b2a6-a6e400e29ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:04.657844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf62fd-2ebb-41a4-b2a6-a6e400e29ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:04.658111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf62fd-2ebb-41a4-b2a6-a6e400e29ee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:04.658834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf62fd-2ebb-41a4-b2a6-a6e400e29ee7 None admin] RBAC: Adding query filter params (role.id=846f40e28e104ec4bc568fe533f9c9a9, scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce, user.id=4c64e6fbe1d44e4d9368dce8bd8c5f5e) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:04.659138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf62fd-2ebb-41a4-b2a6-a6e400e29ee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:04.660564 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf62fd-2ebb-41a4-b2a6-a6e400e29ee7 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:04.672991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 106/205] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:04 2022] GET /identity/v3/role_assignments?user.id=4c64e6fbe1d44e4d9368dce8bd8c5f5e&scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce&role.id=846f40e28e104ec4bc568fe533f9c9a9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:05.470840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d7b3e-3713-402b-bba2-679b95a9d82d None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:05.471343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d7b3e-3713-402b-bba2-679b95a9d82d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:05.471343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d7b3e-3713-402b-bba2-679b95a9d82d None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:05.472641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 100/206] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:05.478409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fdf72-196f-4448-a65d-bc234ce6565f None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:05.479239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fdf72-196f-4448-a65d-bc234ce6565f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:05.479690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fdf72-196f-4448-a65d-bc234ce6565f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:05.498475 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-357fdf72-196f-4448-a65d-bc234ce6565f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:05.500443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-357fdf72-196f-4448-a65d-bc234ce6565f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:05.519747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:05.576610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 107/207] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:05.658868 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302531b-892a-4a78-82ad-386fe66c59f8 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:05.659154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302531b-892a-4a78-82ad-386fe66c59f8 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:05.659602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302531b-892a-4a78-82ad-386fe66c59f8 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:05.661255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 101/208] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:05.667028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f3711-11a2-4b97-ba61-434dcffc67a9 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:05.667405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f3711-11a2-4b97-ba61-434dcffc67a9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:05.667567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f3711-11a2-4b97-ba61-434dcffc67a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:05.690668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ae5f3711-11a2-4b97-ba61-434dcffc67a9 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:05.691879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ae5f3711-11a2-4b97-ba61-434dcffc67a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:05.760633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 108/209] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:05.777410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c5762-0928-4b4a-914b-df13959324cd None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:05.777703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c5762-0928-4b4a-914b-df13959324cd None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:05.778227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c5762-0928-4b4a-914b-df13959324cd None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:05.779745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 102/210] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:05.785719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79afde-aab1-4c8d-a4fb-0f2ec1e11f9e None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:05.785996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79afde-aab1-4c8d-a4fb-0f2ec1e11f9e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:05.786273 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79afde-aab1-4c8d-a4fb-0f2ec1e11f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:05.806230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-fc79afde-aab1-4c8d-a4fb-0f2ec1e11f9e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:05.807484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fc79afde-aab1-4c8d-a4fb-0f2ec1e11f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:05.864874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c5762-0928-4b4a-914b-df13959324cd None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:05.885066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 109/211] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:05.980021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c5762-0928-4b4a-914b-df13959324cd None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:06.015949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28173b8-aaad-4ba2-9c8c-1d703d1970f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:06.016871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28173b8-aaad-4ba2-9c8c-1d703d1970f5 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:06.017323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28173b8-aaad-4ba2-9c8c-1d703d1970f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:06.017762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28173b8-aaad-4ba2-9c8c-1d703d1970f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:06.019079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28173b8-aaad-4ba2-9c8c-1d703d1970f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:06.020726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28173b8-aaad-4ba2-9c8c-1d703d1970f5 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:06.022450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 103/212] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:06.027938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79afde-aab1-4c8d-a4fb-0f2ec1e11f9e None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:06.029278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28173b8-aaad-4ba2-9c8c-1d703d1970f5 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:06.044795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:06.083962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:06.134180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28173b8-aaad-4ba2-9c8c-1d703d1970f5 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:06.148147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79afde-aab1-4c8d-a4fb-0f2ec1e11f9e None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:06.183867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7db276-e5d1-4dac-b582-2e4dfd329163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:06.184995 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7db276-e5d1-4dac-b582-2e4dfd329163 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:06.185403 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7db276-e5d1-4dac-b582-2e4dfd329163 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:06.185867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7db276-e5d1-4dac-b582-2e4dfd329163 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:06.186417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804feda5-b6ce-4fe9-87b8-6f420c021851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:06.187584 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804feda5-b6ce-4fe9-87b8-6f420c021851 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:06.187859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804feda5-b6ce-4fe9-87b8-6f420c021851 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:06.188128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804feda5-b6ce-4fe9-87b8-6f420c021851 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:06.188789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804feda5-b6ce-4fe9-87b8-6f420c021851 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:06.190258 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804feda5-b6ce-4fe9-87b8-6f420c021851 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:06.202623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7db276-e5d1-4dac-b582-2e4dfd329163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-0c7db276-e5d1-4dac-b582-2e4dfd329163 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:06.208294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:06.209525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 104/213] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:06.210303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7db276-e5d1-4dac-b582-2e4dfd329163 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:06.215483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 110/214] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:27:06 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:27:06.221152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804feda5-b6ce-4fe9-87b8-6f420c021851 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:06.305437 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:06.335885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7db276-e5d1-4dac-b582-2e4dfd329163 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:06.346825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804feda5-b6ce-4fe9-87b8-6f420c021851 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:06.351929 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:06.376853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81db3b9-5bb0-4008-8f3f-d488f744ef38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:06.377550 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81db3b9-5bb0-4008-8f3f-d488f744ef38 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:06.377832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81db3b9-5bb0-4008-8f3f-d488f744ef38 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:06.378107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81db3b9-5bb0-4008-8f3f-d488f744ef38 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:06.388156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87b5fc-735d-4e18-b01f-401894501395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:06.388785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b5fc-735d-4e18-b01f-401894501395 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:06.389094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b5fc-735d-4e18-b01f-401894501395 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:06.389370 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b5fc-735d-4e18-b01f-401894501395 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:06.390086 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87b5fc-735d-4e18-b01f-401894501395 None admin] RBAC: Adding query filter params (name=member) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:06.390390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87b5fc-735d-4e18-b01f-401894501395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:06.391500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81db3b9-5bb0-4008-8f3f-d488f744ef38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:06.391803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87b5fc-735d-4e18-b01f-401894501395 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-a81db3b9-5bb0-4008-8f3f-d488f744ef38 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:06.397152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:06.398038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:06.399152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 105/215] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:06.400667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 111/216] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:06.405060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81db3b9-5bb0-4008-8f3f-d488f744ef38 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:06.407486 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87b5fc-735d-4e18-b01f-401894501395 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:06.522948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81db3b9-5bb0-4008-8f3f-d488f744ef38 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:06.525693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87b5fc-735d-4e18-b01f-401894501395 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:06.563960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:06.565461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfd11b-7a1d-449a-b265-392e0d7ddf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:06.566077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfd11b-7a1d-449a-b265-392e0d7ddf8b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:06.566515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfd11b-7a1d-449a-b265-392e0d7ddf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:06.566959 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfd11b-7a1d-449a-b265-392e0d7ddf8b None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:06.567635 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfd11b-7a1d-449a-b265-392e0d7ddf8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:06.567806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705a8b3-4983-4111-8d8f-9f58b67df95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:06.567971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfd11b-7a1d-449a-b265-392e0d7ddf8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:06.568426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705a8b3-4983-4111-8d8f-9f58b67df95c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:06.569078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705a8b3-4983-4111-8d8f-9f58b67df95c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:06.569366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfd11b-7a1d-449a-b265-392e0d7ddf8b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:06.570009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705a8b3-4983-4111-8d8f-9f58b67df95c None admin] PATH_INFO: `/v3/users/49eb82a7b6764fcba1044a496acbea38` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:06.579378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 106/217] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:06.586166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfd11b-7a1d-449a-b265-392e0d7ddf8b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:06.587702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705a8b3-4983-4111-8d8f-9f58b67df95c None admin] RBAC: Authorizing `identity:get_user(user_id=49eb82a7b6764fcba1044a496acbea38)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:06.589608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705a8b3-4983-4111-8d8f-9f58b67df95c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:06.592425 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 112/218] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:06 2022] GET /identity/v3/users/49eb82a7b6764fcba1044a496acbea38 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:06.598944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705a8b3-4983-4111-8d8f-9f58b67df95c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:06.626015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:06.725924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfd11b-7a1d-449a-b265-392e0d7ddf8b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:06.731388 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705a8b3-4983-4111-8d8f-9f58b67df95c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:06.771000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8989fcb-2c06-4dc8-bf7d-6ccc67ddfc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:06.771873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8989fcb-2c06-4dc8-bf7d-6ccc67ddfc3b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:06.772284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8989fcb-2c06-4dc8-bf7d-6ccc67ddfc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:06.772697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8989fcb-2c06-4dc8-bf7d-6ccc67ddfc3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:06.773158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb3a80-e41b-4c79-8c14-90cae92bc578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:06.773808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb3a80-e41b-4c79-8c14-90cae92bc578 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:06.774033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8989fcb-2c06-4dc8-bf7d-6ccc67ddfc3b None admin] RBAC: Adding query filter params (role.id=846f40e28e104ec4bc568fe533f9c9a9, user.id=49eb82a7b6764fcba1044a496acbea38, scope.system=all) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:06.774140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb3a80-e41b-4c79-8c14-90cae92bc578 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:06.774351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb3a80-e41b-4c79-8c14-90cae92bc578 None admin] PATH_INFO: `/v3/users/c6c84e72064c45358de3cd5ba8ff8c7f` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:06.774705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8989fcb-2c06-4dc8-bf7d-6ccc67ddfc3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:06.775596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb3a80-e41b-4c79-8c14-90cae92bc578 None admin] RBAC: Authorizing `identity:get_user(user_id=c6c84e72064c45358de3cd5ba8ff8c7f)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:06.776757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8989fcb-2c06-4dc8-bf7d-6ccc67ddfc3b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:06.777124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb3a80-e41b-4c79-8c14-90cae92bc578 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:06.785035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 107/219] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:06 2022] GET /identity/v3/users/c6c84e72064c45358de3cd5ba8ff8c7f => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:06.786104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb3a80-e41b-4c79-8c14-90cae92bc578 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:06.789288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 113/220] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Oct 18 11:27:06 2022] GET /identity/v3/role_assignments?user.id=49eb82a7b6764fcba1044a496acbea38&scope.system=all&role.id=846f40e28e104ec4bc568fe533f9c9a9 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:06.907585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb3a80-e41b-4c79-8c14-90cae92bc578 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:06.909394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:06.947772 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aded89e-cf38-42ca-9357-3495eb23d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:06.948416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aded89e-cf38-42ca-9357-3495eb23d652 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:06.948696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aded89e-cf38-42ca-9357-3495eb23d652 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:06.949005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aded89e-cf38-42ca-9357-3495eb23d652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:07.033836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aded89e-cf38-42ca-9357-3495eb23d652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:07.035692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aded89e-cf38-42ca-9357-3495eb23d652 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:07.146240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 108/221] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:27:07.152446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8989fcb-2c06-4dc8-bf7d-6ccc67ddfc3b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:07.216894 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:07.269005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8989fcb-2c06-4dc8-bf7d-6ccc67ddfc3b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:07.309067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3577914-d079-40e9-8a2a-7c252d0d32a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:07.309717 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3577914-d079-40e9-8a2a-7c252d0d32a8 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:07.309994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3577914-d079-40e9-8a2a-7c252d0d32a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:07.310266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3577914-d079-40e9-8a2a-7c252d0d32a8 None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:07.311444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3577914-d079-40e9-8a2a-7c252d0d32a8 None admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:07.313525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3577914-d079-40e9-8a2a-7c252d0d32a8 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:07.315202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 114/222] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:27:07 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:07.322057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aded89e-cf38-42ca-9357-3495eb23d652 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:07.324619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:07.438759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aded89e-cf38-42ca-9357-3495eb23d652 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:07.475226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302835b-6823-4ee5-abcd-64c29dddaa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:07.475844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302835b-6823-4ee5-abcd-64c29dddaa2d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:07.476126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302835b-6823-4ee5-abcd-64c29dddaa2d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:07.476406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302835b-6823-4ee5-abcd-64c29dddaa2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:07.477163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302835b-6823-4ee5-abcd-64c29dddaa2d None admin] RBAC: Adding query filter params (role.id=235abd0f332d45e69a86877b4fb95973, scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce, user.id=c6c84e72064c45358de3cd5ba8ff8c7f) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:07.477475 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302835b-6823-4ee5-abcd-64c29dddaa2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:07.478930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302835b-6823-4ee5-abcd-64c29dddaa2d None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:07.485892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 109/223] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:07 2022] GET /identity/v3/role_assignments?user.id=c6c84e72064c45358de3cd5ba8ff8c7f&scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce&role.id=235abd0f332d45e69a86877b4fb95973 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:08.581599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f0a4-4937-4a96-8f4d-30a88f067e99 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:08.582274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f0a4-4937-4a96-8f4d-30a88f067e99 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:08.582456 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f0a4-4937-4a96-8f4d-30a88f067e99 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:08.583791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 115/224] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:08.589482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a25c58-7616-4d57-ba96-9c4734ff4e2c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:08.589920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a25c58-7616-4d57-ba96-9c4734ff4e2c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:08.590038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a25c58-7616-4d57-ba96-9c4734ff4e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:08.609173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-84a25c58-7616-4d57-ba96-9c4734ff4e2c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:08.610387 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-84a25c58-7616-4d57-ba96-9c4734ff4e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:08.668228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44120743-18eb-4786-821d-d81023b47c38 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:08.668510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44120743-18eb-4786-821d-d81023b47c38 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:08.668982 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44120743-18eb-4786-821d-d81023b47c38 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:08.670556 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 116/225] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:27:08.675857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:08.676765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1203b-e2a1-4f9d-8da6-0585160f8058 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:08.677227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1203b-e2a1-4f9d-8da6-0585160f8058 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:08.677711 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1203b-e2a1-4f9d-8da6-0585160f8058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:08.683788 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 110/226] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:08.697202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-90f1203b-e2a1-4f9d-8da6-0585160f8058 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:08.698747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-90f1203b-e2a1-4f9d-8da6-0585160f8058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:08.769154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 117/227] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:27:08.954240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a25c58-7616-4d57-ba96-9c4734ff4e2c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:09.031388 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d33a-5b50-41d2-9101-b3fa681428ec None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.031713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d33a-5b50-41d2-9101-b3fa681428ec None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.031975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d33a-5b50-41d2-9101-b3fa681428ec None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.034038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 118/228] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:27:09.038940 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da7f03-4a92-486a-ab6d-293514cf547f None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.039224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da7f03-4a92-486a-ab6d-293514cf547f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.039496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da7f03-4a92-486a-ab6d-293514cf547f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.057464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-56da7f03-4a92-486a-ab6d-293514cf547f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:09.057755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a25c58-7616-4d57-ba96-9c4734ff4e2c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:09.058647 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-56da7f03-4a92-486a-ab6d-293514cf547f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:09.093423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a1ba0-04f4-461a-aa5b-72a8fb23068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:09.094792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a1ba0-04f4-461a-aa5b-72a8fb23068a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.094792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a1ba0-04f4-461a-aa5b-72a8fb23068a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.094792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a1ba0-04f4-461a-aa5b-72a8fb23068a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.104793 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a1ba0-04f4-461a-aa5b-72a8fb23068a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:09.106125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-466a1ba0-04f4-461a-aa5b-72a8fb23068a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:09.109733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:09.110759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:09.111539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 111/229] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:09.112191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a1ba0-04f4-461a-aa5b-72a8fb23068a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:09.129672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 119/230] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:27:09.130317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da7f03-4a92-486a-ab6d-293514cf547f None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:09.182757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:09.240629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a1ba0-04f4-461a-aa5b-72a8fb23068a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:09.241568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da7f03-4a92-486a-ab6d-293514cf547f None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:09.277176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5973a5-f8ce-4abc-9e71-7d0e9ec3d28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:09.277785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5973a5-f8ce-4abc-9e71-7d0e9ec3d28e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.278061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5973a5-f8ce-4abc-9e71-7d0e9ec3d28e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.278230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a0cd6-9f90-45d0-a148-a3699a3cd7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:09.278337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5973a5-f8ce-4abc-9e71-7d0e9ec3d28e None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.278842 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a0cd6-9f90-45d0-a148-a3699a3cd7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.279009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5973a5-f8ce-4abc-9e71-7d0e9ec3d28e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:09.279108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a0cd6-9f90-45d0-a148-a3699a3cd7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.279272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5973a5-f8ce-4abc-9e71-7d0e9ec3d28e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:09.279362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a0cd6-9f90-45d0-a148-a3699a3cd7b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.280518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a0cd6-9f90-45d0-a148-a3699a3cd7b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:09.280688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5973a5-f8ce-4abc-9e71-7d0e9ec3d28e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:09.282275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a0cd6-9f90-45d0-a148-a3699a3cd7b3 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:09.284180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 112/231] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:09.289704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a0cd6-9f90-45d0-a148-a3699a3cd7b3 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:09.290065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 120/232] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:09.295644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5973a5-f8ce-4abc-9e71-7d0e9ec3d28e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:09.346152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:09.398489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a0cd6-9f90-45d0-a148-a3699a3cd7b3 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:09.406960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5973a5-f8ce-4abc-9e71-7d0e9ec3d28e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:09.425465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:09.434824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e26daf-6f5d-441c-a360-c32656df93b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:09.435433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e26daf-6f5d-441c-a360-c32656df93b9 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.435710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e26daf-6f5d-441c-a360-c32656df93b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.435985 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e26daf-6f5d-441c-a360-c32656df93b9 None admin] PATH_INFO: `/v3/groups` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.437167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e26daf-6f5d-441c-a360-c32656df93b9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:09.438599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e26daf-6f5d-441c-a360-c32656df93b9 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:09.441123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8257092-93a4-49e5-9709-551e26384c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:09.441763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257092-93a4-49e5-9709-551e26384c55 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.442039 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257092-93a4-49e5-9709-551e26384c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.442309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257092-93a4-49e5-9709-551e26384c55 None admin] PATH_INFO: `/v3/users/7034be7e2d4d4572898174cdf640520c` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.454275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 113/233] 10.222.0.17 () {60 vars in 1134 bytes} [Tue Oct 18 11:27:09 2022] POST /identity/v3/groups => generated 229 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:27:09.454275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e26daf-6f5d-441c-a360-c32656df93b9 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:09.460567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8257092-93a4-49e5-9709-551e26384c55 None admin] RBAC: Authorizing `identity:get_user(user_id=7034be7e2d4d4572898174cdf640520c)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:09.460567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8257092-93a4-49e5-9709-551e26384c55 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:09.460567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 121/234] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:09 2022] GET /identity/v3/users/7034be7e2d4d4572898174cdf640520c => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:09.466371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8257092-93a4-49e5-9709-551e26384c55 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:09.582991 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e26daf-6f5d-441c-a360-c32656df93b9 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:09.584248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8257092-93a4-49e5-9709-551e26384c55 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:09.586088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:09.629399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c4cbc-c5e0-439e-921d-768390851ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:09.630104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b542-5fac-478f-888c-91e930443813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:09.630554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c4cbc-c5e0-439e-921d-768390851ded None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.630554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c4cbc-c5e0-439e-921d-768390851ded None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.630699 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c4cbc-c5e0-439e-921d-768390851ded None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.630764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b542-5fac-478f-888c-91e930443813 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.631014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b542-5fac-478f-888c-91e930443813 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.631284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7b542-5fac-478f-888c-91e930443813 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.631592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c4cbc-c5e0-439e-921d-768390851ded None admin] RBAC: Adding query filter params (role.id=235abd0f332d45e69a86877b4fb95973, user.id=7034be7e2d4d4572898174cdf640520c, scope.system=all) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:09.631916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c4cbc-c5e0-439e-921d-768390851ded None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:09.633398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c4cbc-c5e0-439e-921d-768390851ded None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:09.638741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 122/235] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Oct 18 11:27:09 2022] GET /identity/v3/role_assignments?user.id=7034be7e2d4d4572898174cdf640520c&scope.system=all&role.id=235abd0f332d45e69a86877b4fb95973 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:09.644993 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7b542-5fac-478f-888c-91e930443813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-adb7b542-5fac-478f-888c-91e930443813 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:09.649918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:09.651009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:09.651784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 114/236] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:09.657647 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c4cbc-c5e0-439e-921d-768390851ded None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:09.773363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c4cbc-c5e0-439e-921d-768390851ded None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:09.812057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e679288-9c5c-4966-9e5b-4b9690e5ffa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:09.812685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e679288-9c5c-4966-9e5b-4b9690e5ffa6 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.812994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e679288-9c5c-4966-9e5b-4b9690e5ffa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.813266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e679288-9c5c-4966-9e5b-4b9690e5ffa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.813939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e679288-9c5c-4966-9e5b-4b9690e5ffa6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:09.814239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e679288-9c5c-4966-9e5b-4b9690e5ffa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:09.815630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e679288-9c5c-4966-9e5b-4b9690e5ffa6 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:09.825746 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 123/237] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:09.830880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b542-5fac-478f-888c-91e930443813 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:09.880606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:09.937153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7b542-5fac-478f-888c-91e930443813 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:09.971273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8a9ac-11b8-4bda-874f-05f61eda4d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:09.971908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8a9ac-11b8-4bda-874f-05f61eda4d96 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:09.972183 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8a9ac-11b8-4bda-874f-05f61eda4d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:09.972451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8a9ac-11b8-4bda-874f-05f61eda4d96 None admin] PATH_INFO: `/v3/users/49eb82a7b6764fcba1044a496acbea38` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:09.973599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8a9ac-11b8-4bda-874f-05f61eda4d96 None admin] RBAC: Authorizing `identity:get_user(user_id=49eb82a7b6764fcba1044a496acbea38)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:09.975056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8a9ac-11b8-4bda-874f-05f61eda4d96 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:09.976807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 115/238] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:09 2022] GET /identity/v3/users/49eb82a7b6764fcba1044a496acbea38 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:09.982206 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e679288-9c5c-4966-9e5b-4b9690e5ffa6 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:10.013817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:10.089930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e679288-9c5c-4966-9e5b-4b9690e5ffa6 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:10.124677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7659a-9721-4273-8257-a22f2f4cf21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:10.125376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7659a-9721-4273-8257-a22f2f4cf21b None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:10.125648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7659a-9721-4273-8257-a22f2f4cf21b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:10.125915 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7659a-9721-4273-8257-a22f2f4cf21b None admin] PATH_INFO: `/v3/system/users/49eb82a7b6764fcba1044a496acbea38/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:10.127383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7659a-9721-4273-8257-a22f2f4cf21b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=49eb82a7b6764fcba1044a496acbea38, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:10.128817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7659a-9721-4273-8257-a22f2f4cf21b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:10.140349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 124/239] 10.222.0.17 () {58 vars in 1334 bytes} [Tue Oct 18 11:27:09 2022] PUT /identity/v3/system/users/49eb82a7b6764fcba1044a496acbea38/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:11.679840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98603393-7874-4d51-92fc-7adf6f34b4e0 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:11.680358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98603393-7874-4d51-92fc-7adf6f34b4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:11.680358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98603393-7874-4d51-92fc-7adf6f34b4e0 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:11.681899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 116/240] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:11.687235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677aa0-f037-45fa-a79f-994adaa5a305 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:11.687661 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677aa0-f037-45fa-a79f-994adaa5a305 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:11.687831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677aa0-f037-45fa-a79f-994adaa5a305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:11.707324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-83677aa0-f037-45fa-a79f-994adaa5a305 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:11.708567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-83677aa0-f037-45fa-a79f-994adaa5a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:11.783027 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 125/241] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:11.812394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f32aa-f414-4162-a57e-016450ebacb2 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:11.812687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f32aa-f414-4162-a57e-016450ebacb2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:11.812972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f32aa-f414-4162-a57e-016450ebacb2 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:11.814445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 117/242] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:11.819504 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087963-8103-4ef9-9adb-32c08136f422 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:11.819792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087963-8103-4ef9-9adb-32c08136f422 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:11.820062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087963-8103-4ef9-9adb-32c08136f422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:11.838751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d0087963-8103-4ef9-9adb-32c08136f422 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:11.839932 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d0087963-8103-4ef9-9adb-32c08136f422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:11.857641 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:11.911578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 126/243] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:12.057078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f32aa-f414-4162-a57e-016450ebacb2 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:12.161132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f32aa-f414-4162-a57e-016450ebacb2 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:12.177620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0087963-8103-4ef9-9adb-32c08136f422 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:12.183560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:12.209544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee32f4-45cb-4bbc-95d1-a23e71d65deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:12.210287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee32f4-45cb-4bbc-95d1-a23e71d65deb None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:12.210565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee32f4-45cb-4bbc-95d1-a23e71d65deb None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:12.210833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee32f4-45cb-4bbc-95d1-a23e71d65deb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:12.212050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee32f4-45cb-4bbc-95d1-a23e71d65deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:12.213528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee32f4-45cb-4bbc-95d1-a23e71d65deb None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:12.215373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 118/244] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:12.221399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee32f4-45cb-4bbc-95d1-a23e71d65deb None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:12.288960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0087963-8103-4ef9-9adb-32c08136f422 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:12.322718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a397b00-f249-4771-b5d8-957f3f2d621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:12.323269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a397b00-f249-4771-b5d8-957f3f2d621c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:12.323544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a397b00-f249-4771-b5d8-957f3f2d621c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:12.323821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a397b00-f249-4771-b5d8-957f3f2d621c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:12.334289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a397b00-f249-4771-b5d8-957f3f2d621c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:12.335341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee32f4-45cb-4bbc-95d1-a23e71d65deb None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-3a397b00-f249-4771-b5d8-957f3f2d621c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:12.339072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:12.340128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:12.340957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 127/245] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:12.341974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9ca6-5ce2-4973-96b8-ac96e1ffe0cb None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:12.342112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9ca6-5ce2-4973-96b8-ac96e1ffe0cb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:12.342377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9ca6-5ce2-4973-96b8-ac96e1ffe0cb None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:12.343675 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 128/246] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:12.348330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e9ca6-5ce2-4973-96b8-ac96e1ffe0cb None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:12.363220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:12.369698 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77103062-fc75-4dd2-bfd5-b8e2cce6cfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:12.370314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77103062-fc75-4dd2-bfd5-b8e2cce6cfd7 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:12.370589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77103062-fc75-4dd2-bfd5-b8e2cce6cfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:12.370858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77103062-fc75-4dd2-bfd5-b8e2cce6cfd7 None admin] PATH_INFO: `/v3/groups` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:12.371939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77103062-fc75-4dd2-bfd5-b8e2cce6cfd7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:12.373424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77103062-fc75-4dd2-bfd5-b8e2cce6cfd7 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:12.388651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 119/247] 10.222.0.17 () {60 vars in 1134 bytes} [Tue Oct 18 11:27:12 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:27:12.390556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a442e0-7d37-431c-98b9-0ca0a9735c8e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:12.390810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a442e0-7d37-431c-98b9-0ca0a9735c8e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:12.391331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a442e0-7d37-431c-98b9-0ca0a9735c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:12.410735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-17a442e0-7d37-431c-98b9-0ca0a9735c8e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:12.412045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-17a442e0-7d37-431c-98b9-0ca0a9735c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:12.454104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:12.469952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e9ca6-5ce2-4973-96b8-ac96e1ffe0cb None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:12.500417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 120/248] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:12.518933 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbcaee-07c7-48fb-b35f-45e4c4dd0476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:12.519573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbcaee-07c7-48fb-b35f-45e4c4dd0476 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:12.519850 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbcaee-07c7-48fb-b35f-45e4c4dd0476 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:12.520121 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbcaee-07c7-48fb-b35f-45e4c4dd0476 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:12.520818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dbcaee-07c7-48fb-b35f-45e4c4dd0476 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:12.521201 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dbcaee-07c7-48fb-b35f-45e4c4dd0476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:12.522654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dbcaee-07c7-48fb-b35f-45e4c4dd0476 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:12.532449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 129/249] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:12.538278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a442e0-7d37-431c-98b9-0ca0a9735c8e None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:12.644411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a442e0-7d37-431c-98b9-0ca0a9735c8e None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:12.682315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa56b5-83a3-4d25-9ef9-80a3a232abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:12.682945 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa56b5-83a3-4d25-9ef9-80a3a232abe9 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:12.683229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa56b5-83a3-4d25-9ef9-80a3a232abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:12.683488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa56b5-83a3-4d25-9ef9-80a3a232abe9 None admin] PATH_INFO: `/v3/users/7034be7e2d4d4572898174cdf640520c` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:12.684686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa56b5-83a3-4d25-9ef9-80a3a232abe9 None admin] RBAC: Authorizing `identity:get_user(user_id=7034be7e2d4d4572898174cdf640520c)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:12.686241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa56b5-83a3-4d25-9ef9-80a3a232abe9 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:12.688586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 121/250] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:12 2022] GET /identity/v3/users/7034be7e2d4d4572898174cdf640520c => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:12.694497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbcaee-07c7-48fb-b35f-45e4c4dd0476 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:12.773874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa56b5-83a3-4d25-9ef9-80a3a232abe9 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:12.783811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:12.816730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbcaee-07c7-48fb-b35f-45e4c4dd0476 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:12.832489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:12.860678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4953da-6d38-417e-9fdb-45c31cef1af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:12.861633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4953da-6d38-417e-9fdb-45c31cef1af5 None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:12.862057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4953da-6d38-417e-9fdb-45c31cef1af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:12.862497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4953da-6d38-417e-9fdb-45c31cef1af5 None admin] PATH_INFO: `/v3/system/users/7034be7e2d4d4572898174cdf640520c/roles/235abd0f332d45e69a86877b4fb95973` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:12.864135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4953da-6d38-417e-9fdb-45c31cef1af5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7034be7e2d4d4572898174cdf640520c, role_id=235abd0f332d45e69a86877b4fb95973)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:12.866068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4953da-6d38-417e-9fdb-45c31cef1af5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:12.876867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 130/251] 10.222.0.17 () {58 vars in 1334 bytes} [Tue Oct 18 11:27:12 2022] PUT /identity/v3/system/users/7034be7e2d4d4572898174cdf640520c/roles/235abd0f332d45e69a86877b4fb95973 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:12.925963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa56b5-83a3-4d25-9ef9-80a3a232abe9 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:12.971655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf313c3-fd7a-4ba2-95bf-0ad029cda066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:12.972309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf313c3-fd7a-4ba2-95bf-0ad029cda066 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:12.972585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf313c3-fd7a-4ba2-95bf-0ad029cda066 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:12.972874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf313c3-fd7a-4ba2-95bf-0ad029cda066 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:12.991845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf313c3-fd7a-4ba2-95bf-0ad029cda066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-5bf313c3-fd7a-4ba2-95bf-0ad029cda066 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:12.996909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:12.997817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:12.998591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 122/252] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:13.004182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4953da-6d38-417e-9fdb-45c31cef1af5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:13.134243 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4953da-6d38-417e-9fdb-45c31cef1af5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:13.174549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8493af-2988-4d69-9770-320ac1f90391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:13.175599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8493af-2988-4d69-9770-320ac1f90391 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:13.176130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8493af-2988-4d69-9770-320ac1f90391 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:13.176600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8493af-2988-4d69-9770-320ac1f90391 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:13.177513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8493af-2988-4d69-9770-320ac1f90391 None admin] RBAC: Adding query filter params (name=member) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:13.178041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8493af-2988-4d69-9770-320ac1f90391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:13.179631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8493af-2988-4d69-9770-320ac1f90391 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:13.185886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:13.190779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 131/253] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:13.196685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf313c3-fd7a-4ba2-95bf-0ad029cda066 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:13.317611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:13.327665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf313c3-fd7a-4ba2-95bf-0ad029cda066 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:13.364733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a158cb6-e308-4797-96c4-e11d3b7f4365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:13.365405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a158cb6-e308-4797-96c4-e11d3b7f4365 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:13.365679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a158cb6-e308-4797-96c4-e11d3b7f4365 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:13.365943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a158cb6-e308-4797-96c4-e11d3b7f4365 None admin] PATH_INFO: `/v3/users/49eb82a7b6764fcba1044a496acbea38` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:13.367079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a158cb6-e308-4797-96c4-e11d3b7f4365 None admin] RBAC: Authorizing `identity:get_user(user_id=49eb82a7b6764fcba1044a496acbea38)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:13.368521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a158cb6-e308-4797-96c4-e11d3b7f4365 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:13.370544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 123/254] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:13 2022] GET /identity/v3/users/49eb82a7b6764fcba1044a496acbea38 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:13.375991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8493af-2988-4d69-9770-320ac1f90391 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:13.510331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8493af-2988-4d69-9770-320ac1f90391 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:13.553319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a18808-b165-432c-9823-2b8c21694fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:13.553934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a18808-b165-432c-9823-2b8c21694fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:13.554205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a18808-b165-432c-9823-2b8c21694fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:13.554473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a18808-b165-432c-9823-2b8c21694fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:13.555199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a18808-b165-432c-9823-2b8c21694fa0 None admin] RBAC: Adding query filter params (role.id=846f40e28e104ec4bc568fe533f9c9a9, user.id=49eb82a7b6764fcba1044a496acbea38, scope.system=all) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:13.555566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a18808-b165-432c-9823-2b8c21694fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:13.557040 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a18808-b165-432c-9823-2b8c21694fa0 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:13.572383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 132/255] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Oct 18 11:27:13 2022] GET /identity/v3/role_assignments?user.id=49eb82a7b6764fcba1044a496acbea38&scope.system=all&role.id=846f40e28e104ec4bc568fe533f9c9a9 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:14.894569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49402d8f-a1d2-4aaa-a6e2-3a4e9cd40b13 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:14.895087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49402d8f-a1d2-4aaa-a6e2-3a4e9cd40b13 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:14.895087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49402d8f-a1d2-4aaa-a6e2-3a4e9cd40b13 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:14.896447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 124/256] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:14.902186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeca62-313f-42ca-bebc-b5602c4d7971 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:14.902771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeca62-313f-42ca-bebc-b5602c4d7971 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:14.902771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeca62-313f-42ca-bebc-b5602c4d7971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:14.929953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3abeca62-313f-42ca-bebc-b5602c4d7971 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:14.931160 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3abeca62-313f-42ca-bebc-b5602c4d7971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:15.003545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 133/257] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 18 11:27:15.553598 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49402d8f-a1d2-4aaa-a6e2-3a4e9cd40b13 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:15.569218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49402d8f-a1d2-4aaa-a6e2-3a4e9cd40b13 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:15.573780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69572d17-dd7e-4e31-9431-6465bf88647e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:15.574425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69572d17-dd7e-4e31-9431-6465bf88647e admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:15.574706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69572d17-dd7e-4e31-9431-6465bf88647e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:15.574978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69572d17-dd7e-4e31-9431-6465bf88647e admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:15.591786 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69572d17-dd7e-4e31-9431-6465bf88647e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-69572d17-dd7e-4e31-9431-6465bf88647e admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:15.596852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:15.597756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:15.599113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 125/258] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Oct 18 11:27:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:15.603792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeca62-313f-42ca-bebc-b5602c4d7971 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:15.610059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:15.614797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeca62-313f-42ca-bebc-b5602c4d7971 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:15.622935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05d0c6-6425-4e3c-bd5d-6c9a87a14ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:15.623567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05d0c6-6425-4e3c-bd5d-6c9a87a14ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:15.624408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05d0c6-6425-4e3c-bd5d-6c9a87a14ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:15.624681 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05d0c6-6425-4e3c-bd5d-6c9a87a14ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:15.625401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05d0c6-6425-4e3c-bd5d-6c9a87a14ee9 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:15.625706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05d0c6-6425-4e3c-bd5d-6c9a87a14ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:15.627534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WARNING py.warnings [None req-0e05d0c6-6425-4e3c-bd5d-6c9a87a14ee9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:27:15.627534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: warnings.warn(msg) Oct 18 11:27:15.627534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: Oct 18 11:27:15.628474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05d0c6-6425-4e3c-bd5d-6c9a87a14ee9 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:15.642373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e82104-ebdf-4a6c-b43d-2ca4b95abb38 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:15.642678 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e82104-ebdf-4a6c-b43d-2ca4b95abb38 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:15.642928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e82104-ebdf-4a6c-b43d-2ca4b95abb38 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:15.644518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 126/259] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:15.644748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 134/260] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:27:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:15.649861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c6da8-ea99-4aa1-8189-c86860cad016 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:15.650155 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c6da8-ea99-4aa1-8189-c86860cad016 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:15.650438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c6da8-ea99-4aa1-8189-c86860cad016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:15.654278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05d0c6-6425-4e3c-bd5d-6c9a87a14ee9 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:15.679388 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05d0c6-6425-4e3c-bd5d-6c9a87a14ee9 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:15.682043 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e69c6da8-ea99-4aa1-8189-c86860cad016 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:15.683244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e69c6da8-ea99-4aa1-8189-c86860cad016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:15.698207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0781ca-b2a1-47a5-b256-e1af4eb03616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:15.698872 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0781ca-b2a1-47a5-b256-e1af4eb03616 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:15.699149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0781ca-b2a1-47a5-b256-e1af4eb03616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:15.699419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0781ca-b2a1-47a5-b256-e1af4eb03616 admin admin] PATH_INFO: `/v3/groups/53d323bac77c4ffeab72dc3e5bed383e` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:15.710214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0781ca-b2a1-47a5-b256-e1af4eb03616 admin admin] RBAC: Authorizing `identity:get_group(group_id=53d323bac77c4ffeab72dc3e5bed383e)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:15.711268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WARNING py.warnings [None req-8d0781ca-b2a1-47a5-b256-e1af4eb03616 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:27:15.711268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: warnings.warn(msg) Oct 18 11:27:15.711268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: Oct 18 11:27:15.712513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0781ca-b2a1-47a5-b256-e1af4eb03616 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:15.714281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 135/261] 10.222.0.17 () {56 vars in 1201 bytes} [Tue Oct 18 11:27:15 2022] GET /identity/v3/groups/53d323bac77c4ffeab72dc3e5bed383e => generated 226 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:15.724600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0781ca-b2a1-47a5-b256-e1af4eb03616 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:15.740197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0781ca-b2a1-47a5-b256-e1af4eb03616 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:15.748244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b1e18-85e4-4aee-af35-e21102212f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:15.749504 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b1e18-85e4-4aee-af35-e21102212f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:15.749788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b1e18-85e4-4aee-af35-e21102212f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:15.750061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b1e18-85e4-4aee-af35-e21102212f61 admin admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:15.751225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b1e18-85e4-4aee-af35-e21102212f61 admin admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:15.753505 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b1e18-85e4-4aee-af35-e21102212f61 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:15.755519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 136/262] 10.222.0.17 () {56 vars in 1207 bytes} [Tue Oct 18 11:27:15 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:15.769536 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b1e18-85e4-4aee-af35-e21102212f61 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:15.783125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b1e18-85e4-4aee-af35-e21102212f61 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:15.798714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c02eb-92f6-4614-b3ba-cad57329d527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:15.799401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c02eb-92f6-4614-b3ba-cad57329d527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:15.799659 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c02eb-92f6-4614-b3ba-cad57329d527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:15.799930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c02eb-92f6-4614-b3ba-cad57329d527 admin admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/groups/53d323bac77c4ffeab72dc3e5bed383e/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:15.802042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c02eb-92f6-4614-b3ba-cad57329d527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4a0bd34a554747b58ff7dc25238f38, group_id=53d323bac77c4ffeab72dc3e5bed383e, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:15.803095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WARNING py.warnings [None req-4d0c02eb-92f6-4614-b3ba-cad57329d527 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:27:15.803095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: warnings.warn(msg) Oct 18 11:27:15.803095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: Oct 18 11:27:15.805225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c02eb-92f6-4614-b3ba-cad57329d527 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:15.826956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 127/263] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:15.832573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 137/264] 10.222.0.17 () {58 vars in 1463 bytes} [Tue Oct 18 11:27:15 2022] PUT /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/groups/53d323bac77c4ffeab72dc3e5bed383e/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:16.053416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0585ee7-075d-4f89-baa6-358069181dfc None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.054089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0585ee7-075d-4f89-baa6-358069181dfc None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.054089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0585ee7-075d-4f89-baa6-358069181dfc None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.055638 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 128/265] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:16.066572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028b3a3-567e-4733-9d91-c59cc822d2d1 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.067206 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028b3a3-567e-4733-9d91-c59cc822d2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.067206 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028b3a3-567e-4733-9d91-c59cc822d2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.095823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-2028b3a3-567e-4733-9d91-c59cc822d2d1 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:16.097077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2028b3a3-567e-4733-9d91-c59cc822d2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:16.121655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bf1df-7efd-46cc-ac52-89972ac6b120 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.121963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bf1df-7efd-46cc-ac52-89972ac6b120 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.122199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bf1df-7efd-46cc-ac52-89972ac6b120 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.123599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 129/266] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:16.138642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace7ce3-53f3-4f9a-a747-452108cf32b6 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.138999 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace7ce3-53f3-4f9a-a747-452108cf32b6 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.139174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace7ce3-53f3-4f9a-a747-452108cf32b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.186867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-2ace7ce3-53f3-4f9a-a747-452108cf32b6 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:16.188056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2ace7ce3-53f3-4f9a-a747-452108cf32b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:16.222819 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 138/267] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 18 11:27:16.261355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028b3a3-567e-4733-9d91-c59cc822d2d1 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:16.289267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 130/268] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 384 bytes (3 switches on core 0) Oct 18 11:27:16.416755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028b3a3-567e-4733-9d91-c59cc822d2d1 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:16.438776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:16.462660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6331af-3838-4f26-94a0-af9b1cd736f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:16.463714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6331af-3838-4f26-94a0-af9b1cd736f3 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.464184 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6331af-3838-4f26-94a0-af9b1cd736f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.464643 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6331af-3838-4f26-94a0-af9b1cd736f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.476627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6331af-3838-4f26-94a0-af9b1cd736f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-3c6331af-3838-4f26-94a0-af9b1cd736f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:16.484832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:16.485821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 18 11:27:16.486590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:16.489545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 139/269] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:16.495938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace7ce3-53f3-4f9a-a747-452108cf32b6 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:16.542117 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:16.639070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace7ce3-53f3-4f9a-a747-452108cf32b6 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:16.687458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276ae50-4613-4678-8f9e-f32d8c5465de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:16.688008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276ae50-4613-4678-8f9e-f32d8c5465de None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.688286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276ae50-4613-4678-8f9e-f32d8c5465de None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.688555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276ae50-4613-4678-8f9e-f32d8c5465de None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.689262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276ae50-4613-4678-8f9e-f32d8c5465de None admin] RBAC: Adding query filter params (name=reader) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:16.689563 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276ae50-4613-4678-8f9e-f32d8c5465de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:16.690968 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276ae50-4613-4678-8f9e-f32d8c5465de None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:16.700486 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 131/270] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:16.706858 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6331af-3838-4f26-94a0-af9b1cd736f3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:16.717783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276ae50-4613-4678-8f9e-f32d8c5465de None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:16.737030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276ae50-4613-4678-8f9e-f32d8c5465de None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:16.747673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b796d87-e29f-4d4b-b099-b2fd1d69b56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:16.748312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b796d87-e29f-4d4b-b099-b2fd1d69b56f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.748584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b796d87-e29f-4d4b-b099-b2fd1d69b56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.749417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b796d87-e29f-4d4b-b099-b2fd1d69b56f admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.773703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b796d87-e29f-4d4b-b099-b2fd1d69b56f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-5b796d87-e29f-4d4b-b099-b2fd1d69b56f admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:16.781427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:16.782370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:27:16.783153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:16.786267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 132/271] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Oct 18 11:27:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 69 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:16.794553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b796d87-e29f-4d4b-b099-b2fd1d69b56f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:16.816707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b796d87-e29f-4d4b-b099-b2fd1d69b56f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:16.829119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5418334-b8fd-44ba-85e6-ef74265f92fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:16.829814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5418334-b8fd-44ba-85e6-ef74265f92fc admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.830095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5418334-b8fd-44ba-85e6-ef74265f92fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.830420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5418334-b8fd-44ba-85e6-ef74265f92fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.832159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5418334-b8fd-44ba-85e6-ef74265f92fc admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:16.832480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5418334-b8fd-44ba-85e6-ef74265f92fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:16.834570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WARNING py.warnings [None req-e5418334-b8fd-44ba-85e6-ef74265f92fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:27:16.834570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: warnings.warn(msg) Oct 18 11:27:16.834570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: Oct 18 11:27:16.835505 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5418334-b8fd-44ba-85e6-ef74265f92fc admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:16.855507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 133/272] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Oct 18 11:27:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:16.856136 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5418334-b8fd-44ba-85e6-ef74265f92fc admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:16.876795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5418334-b8fd-44ba-85e6-ef74265f92fc admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:16.887247 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119106af-7276-4f03-9624-899a145c28b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:16.888437 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119106af-7276-4f03-9624-899a145c28b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.888880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119106af-7276-4f03-9624-899a145c28b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.889056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119106af-7276-4f03-9624-899a145c28b9 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.898788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6331af-3838-4f26-94a0-af9b1cd736f3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:16.904980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119106af-7276-4f03-9624-899a145c28b9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-119106af-7276-4f03-9624-899a145c28b9 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:16.910952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:16.911836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:16.912633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:16.920866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 134/273] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Oct 18 11:27:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 59 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:16.920866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119106af-7276-4f03-9624-899a145c28b9 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:16.927557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119106af-7276-4f03-9624-899a145c28b9 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:16.936014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4e784-ae58-4e6b-bfe7-a67cda9f9017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:16.937323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4e784-ae58-4e6b-bfe7-a67cda9f9017 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.937659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4e784-ae58-4e6b-bfe7-a67cda9f9017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.937939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4e784-ae58-4e6b-bfe7-a67cda9f9017 admin admin] PATH_INFO: `/v3/groups/53d323bac77c4ffeab72dc3e5bed383e` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.939106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4e784-ae58-4e6b-bfe7-a67cda9f9017 admin admin] RBAC: Authorizing `identity:get_group(group_id=53d323bac77c4ffeab72dc3e5bed383e)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:16.940187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WARNING py.warnings [None req-b3f4e784-ae58-4e6b-bfe7-a67cda9f9017 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:27:16.940187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: warnings.warn(msg) Oct 18 11:27:16.940187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: Oct 18 11:27:16.944864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4e784-ae58-4e6b-bfe7-a67cda9f9017 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:16.944864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 135/274] 10.222.0.17 () {56 vars in 1201 bytes} [Tue Oct 18 11:27:16 2022] GET /identity/v3/groups/53d323bac77c4ffeab72dc3e5bed383e => generated 226 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:16.944864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4e784-ae58-4e6b-bfe7-a67cda9f9017 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:16.965700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4e784-ae58-4e6b-bfe7-a67cda9f9017 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:16.972491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62911f99-d435-4f17-94fb-b257993bbe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:16.973497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62911f99-d435-4f17-94fb-b257993bbe33 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.973776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62911f99-d435-4f17-94fb-b257993bbe33 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.974615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62911f99-d435-4f17-94fb-b257993bbe33 None admin] PATH_INFO: `/v3/users/7034be7e2d4d4572898174cdf640520c` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.975278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb3c7e-b779-4760-8ca0-c88ce0b5a99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:16.975892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb3c7e-b779-4760-8ca0-c88ce0b5a99e admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:16.976165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb3c7e-b779-4760-8ca0-c88ce0b5a99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:16.977045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb3c7e-b779-4760-8ca0-c88ce0b5a99e admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:16.977851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb3c7e-b779-4760-8ca0-c88ce0b5a99e admin admin] RBAC: Adding query filter params (name=member) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:16.978156 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb3c7e-b779-4760-8ca0-c88ce0b5a99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:16.979310 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62911f99-d435-4f17-94fb-b257993bbe33 None admin] RBAC: Authorizing `identity:get_user(user_id=7034be7e2d4d4572898174cdf640520c)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:16.980922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62911f99-d435-4f17-94fb-b257993bbe33 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:16.983230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb3c7e-b779-4760-8ca0-c88ce0b5a99e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:16.984628 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 140/275] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:27:16 2022] GET /identity/v3/users/7034be7e2d4d4572898174cdf640520c => generated 293 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:16.986638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62911f99-d435-4f17-94fb-b257993bbe33 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:17.009988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 136/276] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:27:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:17.012799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb3c7e-b779-4760-8ca0-c88ce0b5a99e admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:17.015970 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62911f99-d435-4f17-94fb-b257993bbe33 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:17.027226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330b70e-4e0a-4729-83a2-ef30afe10129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:17.027928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330b70e-4e0a-4729-83a2-ef30afe10129 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:17.028206 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330b70e-4e0a-4729-83a2-ef30afe10129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:17.028525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330b70e-4e0a-4729-83a2-ef30afe10129 admin admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:17.029771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330b70e-4e0a-4729-83a2-ef30afe10129 admin admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:17.032809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330b70e-4e0a-4729-83a2-ef30afe10129 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:17.038180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 141/277] 10.222.0.17 () {56 vars in 1207 bytes} [Tue Oct 18 11:27:16 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:17.039274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330b70e-4e0a-4729-83a2-ef30afe10129 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:17.051855 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330b70e-4e0a-4729-83a2-ef30afe10129 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:17.056989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9aa97a-a3fa-48b3-a942-3bc571d993c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:17.058244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aa97a-a3fa-48b3-a942-3bc571d993c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:17.058579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aa97a-a3fa-48b3-a942-3bc571d993c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:17.058856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aa97a-a3fa-48b3-a942-3bc571d993c8 admin admin] PATH_INFO: `/v3/groups/53d323bac77c4ffeab72dc3e5bed383e` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:17.064886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9aa97a-a3fa-48b3-a942-3bc571d993c8 admin admin] RBAC: Authorizing `identity:get_group(group_id=53d323bac77c4ffeab72dc3e5bed383e)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:17.067539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9aa97a-a3fa-48b3-a942-3bc571d993c8 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:17.069430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 142/278] 10.222.0.17 () {56 vars in 1201 bytes} [Tue Oct 18 11:27:17 2022] GET /identity/v3/groups/53d323bac77c4ffeab72dc3e5bed383e => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:17.070229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9aa97a-a3fa-48b3-a942-3bc571d993c8 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:17.082267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9aa97a-a3fa-48b3-a942-3bc571d993c8 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:17.088342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1fb95-2d36-4bae-86d4-cc792cc0f433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:17.089269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1fb95-2d36-4bae-86d4-cc792cc0f433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:17.089609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1fb95-2d36-4bae-86d4-cc792cc0f433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:17.089889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1fb95-2d36-4bae-86d4-cc792cc0f433 admin admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/groups/53d323bac77c4ffeab72dc3e5bed383e/roles/04b8e62238cb4198928fd327ef78e6ae` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:17.092662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1fb95-2d36-4bae-86d4-cc792cc0f433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4a0bd34a554747b58ff7dc25238f38, group_id=53d323bac77c4ffeab72dc3e5bed383e, role_id=04b8e62238cb4198928fd327ef78e6ae)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:17.095813 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1fb95-2d36-4bae-86d4-cc792cc0f433 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:17.129594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:17.132158 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 143/279] 10.222.0.17 () {58 vars in 1463 bytes} [Tue Oct 18 11:27:17 2022] PUT /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38/groups/53d323bac77c4ffeab72dc3e5bed383e/roles/04b8e62238cb4198928fd327ef78e6ae => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:17.134174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1fb95-2d36-4bae-86d4-cc792cc0f433 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:17.205343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:17.267562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb3c7e-b779-4760-8ca0-c88ce0b5a99e admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:17.291158 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1fb95-2d36-4bae-86d4-cc792cc0f433 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:17.303279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc38177-0696-4f38-93ad-2484cc16b38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:17.303943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38177-0696-4f38-93ad-2484cc16b38a admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:17.304222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38177-0696-4f38-93ad-2484cc16b38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:17.304502 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38177-0696-4f38-93ad-2484cc16b38a admin admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:17.309969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc38177-0696-4f38-93ad-2484cc16b38a admin admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:17.312679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc38177-0696-4f38-93ad-2484cc16b38a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:17.314814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 144/280] 10.222.0.17 () {56 vars in 1207 bytes} [Tue Oct 18 11:27:17 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:17.320984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc38177-0696-4f38-93ad-2484cc16b38a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:17.331894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b126b-edd2-4176-b412-85f5d6805d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:17.332785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b126b-edd2-4176-b412-85f5d6805d54 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:17.333250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b126b-edd2-4176-b412-85f5d6805d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:17.333535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b126b-edd2-4176-b412-85f5d6805d54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:17.334387 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b126b-edd2-4176-b412-85f5d6805d54 None admin] RBAC: Adding query filter params (role.id=235abd0f332d45e69a86877b4fb95973, user.id=7034be7e2d4d4572898174cdf640520c, scope.system=all) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:17.334705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b126b-edd2-4176-b412-85f5d6805d54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:17.335351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc38177-0696-4f38-93ad-2484cc16b38a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:17.336170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b126b-edd2-4176-b412-85f5d6805d54 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:17.352480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b74a7a-24c0-4378-8eb1-7719551bf6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:17.353207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b74a7a-24c0-4378-8eb1-7719551bf6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:17.353488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b74a7a-24c0-4378-8eb1-7719551bf6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:17.353760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b74a7a-24c0-4378-8eb1-7719551bf6c0 admin admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/groups/53d323bac77c4ffeab72dc3e5bed383e/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:17.356126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 137/281] 10.222.0.17 () {56 vars in 1308 bytes} [Tue Oct 18 11:27:17 2022] GET /identity/v3/role_assignments?user.id=7034be7e2d4d4572898174cdf640520c&scope.system=all&role.id=235abd0f332d45e69a86877b4fb95973 => generated 510 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:17.357041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b74a7a-24c0-4378-8eb1-7719551bf6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1073f38ce4ec449ba0477f2e9e1786ce, group_id=53d323bac77c4ffeab72dc3e5bed383e, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:17.360323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b74a7a-24c0-4378-8eb1-7719551bf6c0 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:17.387019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 145/282] 10.222.0.17 () {58 vars in 1463 bytes} [Tue Oct 18 11:27:17 2022] PUT /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/groups/53d323bac77c4ffeab72dc3e5bed383e/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:17.666948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bc1a3-5f06-4820-bf30-06b70ac2a1f0 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:17.667352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bc1a3-5f06-4820-bf30-06b70ac2a1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:17.667525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bc1a3-5f06-4820-bf30-06b70ac2a1f0 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:17.669170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 138/283] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:17.676139 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64c521-4003-4804-a8fe-03738627e7ee None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:17.676452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64c521-4003-4804-a8fe-03738627e7ee None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:17.676701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64c521-4003-4804-a8fe-03738627e7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:17.700569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9b64c521-4003-4804-a8fe-03738627e7ee None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:17.701910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9b64c521-4003-4804-a8fe-03738627e7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:17.769785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 146/284] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 18 11:27:17.947088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1300567-97b0-4dec-8a8b-a5a198764b40 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:17.947291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1300567-97b0-4dec-8a8b-a5a198764b40 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:17.947544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1300567-97b0-4dec-8a8b-a5a198764b40 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:17.949265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 139/285] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:17.955097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752c3f-c53b-4321-a21f-006ddf8da8c4 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:17.955386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752c3f-c53b-4321-a21f-006ddf8da8c4 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:17.955665 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752c3f-c53b-4321-a21f-006ddf8da8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:17.992266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-35752c3f-c53b-4321-a21f-006ddf8da8c4 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:17.993492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-35752c3f-c53b-4321-a21f-006ddf8da8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:18.004257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 147/286] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 18 11:27:18.174208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1300567-97b0-4dec-8a8b-a5a198764b40 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:18.198164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1300567-97b0-4dec-8a8b-a5a198764b40 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:18.203177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41517dec-569c-4dbb-862a-7f8ce3cc4907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:18.203817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41517dec-569c-4dbb-862a-7f8ce3cc4907 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:18.204098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41517dec-569c-4dbb-862a-7f8ce3cc4907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:18.204374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41517dec-569c-4dbb-862a-7f8ce3cc4907 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:18.215694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41517dec-569c-4dbb-862a-7f8ce3cc4907 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-41517dec-569c-4dbb-862a-7f8ce3cc4907 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:18.225225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:18.226095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:18.226874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:18.229134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 140/287] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Oct 18 11:27:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:18.234997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35752c3f-c53b-4321-a21f-006ddf8da8c4 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:18.250635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35752c3f-c53b-4321-a21f-006ddf8da8c4 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:18.261104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2080c7-601a-44a4-8520-079fefdb8b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:18.261726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2080c7-601a-44a4-8520-079fefdb8b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:18.262005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2080c7-601a-44a4-8520-079fefdb8b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:18.262275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2080c7-601a-44a4-8520-079fefdb8b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:18.262958 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2080c7-601a-44a4-8520-079fefdb8b3f admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:18.263258 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2080c7-601a-44a4-8520-079fefdb8b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:18.265523 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2080c7-601a-44a4-8520-079fefdb8b3f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:18.281141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 148/288] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Oct 18 11:27:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:18.286464 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41517dec-569c-4dbb-862a-7f8ce3cc4907 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:18.296737 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41517dec-569c-4dbb-862a-7f8ce3cc4907 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:18.309530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeb5bb-65df-4fc7-add8-b3ee41f49eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:18.310486 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeb5bb-65df-4fc7-add8-b3ee41f49eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:18.311001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeb5bb-65df-4fc7-add8-b3ee41f49eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:18.312006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffeb5bb-65df-4fc7-add8-b3ee41f49eb4 admin admin] PATH_INFO: `/v3/groups/52d3ec9bcb0641df8c8f39185bbca677` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:18.321488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeb5bb-65df-4fc7-add8-b3ee41f49eb4 admin admin] RBAC: Authorizing `identity:get_group(group_id=52d3ec9bcb0641df8c8f39185bbca677)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:18.323714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffeb5bb-65df-4fc7-add8-b3ee41f49eb4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:18.325964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 141/289] 10.222.0.17 () {56 vars in 1201 bytes} [Tue Oct 18 11:27:18 2022] GET /identity/v3/groups/52d3ec9bcb0641df8c8f39185bbca677 => generated 229 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:18.331744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2080c7-601a-44a4-8520-079fefdb8b3f admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:18.346161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2080c7-601a-44a4-8520-079fefdb8b3f admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:18.352980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aad578-79f0-49fb-95c0-b14298fff622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:18.353882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aad578-79f0-49fb-95c0-b14298fff622 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:18.354359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aad578-79f0-49fb-95c0-b14298fff622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:18.354824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aad578-79f0-49fb-95c0-b14298fff622 admin admin] PATH_INFO: `/v3/projects/4c023e59351c4a06998c34820bba9111` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:18.356197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aad578-79f0-49fb-95c0-b14298fff622 admin admin] RBAC: Authorizing `identity:get_project(project_id=4c023e59351c4a06998c34820bba9111)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:18.358290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aad578-79f0-49fb-95c0-b14298fff622 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:18.361210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 149/290] 10.222.0.17 () {56 vars in 1207 bytes} [Tue Oct 18 11:27:18 2022] GET /identity/v3/projects/4c023e59351c4a06998c34820bba9111 => generated 344 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:18.366054 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeb5bb-65df-4fc7-add8-b3ee41f49eb4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:18.377074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffeb5bb-65df-4fc7-add8-b3ee41f49eb4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:18.385071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b061f5-c78f-4e9e-ac31-0fdd1b9f826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:18.386197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b061f5-c78f-4e9e-ac31-0fdd1b9f826f admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:18.386678 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b061f5-c78f-4e9e-ac31-0fdd1b9f826f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:18.387163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b061f5-c78f-4e9e-ac31-0fdd1b9f826f admin admin] PATH_INFO: `/v3/projects/4c023e59351c4a06998c34820bba9111/groups/52d3ec9bcb0641df8c8f39185bbca677/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:18.389907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b061f5-c78f-4e9e-ac31-0fdd1b9f826f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c023e59351c4a06998c34820bba9111, group_id=52d3ec9bcb0641df8c8f39185bbca677, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:18.391152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WARNING py.warnings [None req-60b061f5-c78f-4e9e-ac31-0fdd1b9f826f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:27:18.391152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: warnings.warn(msg) Oct 18 11:27:18.391152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: Oct 18 11:27:18.393521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b061f5-c78f-4e9e-ac31-0fdd1b9f826f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:18.416081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 142/291] 10.222.0.17 () {58 vars in 1463 bytes} [Tue Oct 18 11:27:18 2022] PUT /identity/v3/projects/4c023e59351c4a06998c34820bba9111/groups/52d3ec9bcb0641df8c8f39185bbca677/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:18.469347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aad578-79f0-49fb-95c0-b14298fff622 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:18.480397 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aad578-79f0-49fb-95c0-b14298fff622 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:18.558923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee346f36-f15e-49a5-af7a-4ea85fa86ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:18.559935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee346f36-f15e-49a5-af7a-4ea85fa86ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:18.560413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee346f36-f15e-49a5-af7a-4ea85fa86ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:18.561118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee346f36-f15e-49a5-af7a-4ea85fa86ce9 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:18.576660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee346f36-f15e-49a5-af7a-4ea85fa86ce9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-ee346f36-f15e-49a5-af7a-4ea85fa86ce9 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:18.582374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:18.583332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:27:18.584115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:18.586696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 150/292] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Oct 18 11:27:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 118 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:18.593280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b061f5-c78f-4e9e-ac31-0fdd1b9f826f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:18.604732 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b061f5-c78f-4e9e-ac31-0fdd1b9f826f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:18.611214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b8735-82da-4751-b092-24213ef3ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:18.612253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b8735-82da-4751-b092-24213ef3ba3f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:18.612787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b8735-82da-4751-b092-24213ef3ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:18.613345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b8735-82da-4751-b092-24213ef3ba3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:18.614577 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b8735-82da-4751-b092-24213ef3ba3f admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:18.615081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b8735-82da-4751-b092-24213ef3ba3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:18.617004 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b8735-82da-4751-b092-24213ef3ba3f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:18.628163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 143/293] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Oct 18 11:27:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:18.634384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee346f36-f15e-49a5-af7a-4ea85fa86ce9 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:18.646783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee346f36-f15e-49a5-af7a-4ea85fa86ce9 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:18.652482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd57ab-3047-4c79-9e3f-01a377b99b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:18.653585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd57ab-3047-4c79-9e3f-01a377b99b0a admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:18.654070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd57ab-3047-4c79-9e3f-01a377b99b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:18.654574 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd57ab-3047-4c79-9e3f-01a377b99b0a admin admin] PATH_INFO: `/v3/groups/53d323bac77c4ffeab72dc3e5bed383e` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:18.656225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd57ab-3047-4c79-9e3f-01a377b99b0a admin admin] RBAC: Authorizing `identity:get_group(group_id=53d323bac77c4ffeab72dc3e5bed383e)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:18.658608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd57ab-3047-4c79-9e3f-01a377b99b0a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:18.660650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 151/294] 10.222.0.17 () {56 vars in 1201 bytes} [Tue Oct 18 11:27:18 2022] GET /identity/v3/groups/53d323bac77c4ffeab72dc3e5bed383e => generated 226 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:18.666868 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b8735-82da-4751-b092-24213ef3ba3f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:18.679105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b8735-82da-4751-b092-24213ef3ba3f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:18.685385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9060853-477b-4225-a0bd-6367e92aa95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:18.687302 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060853-477b-4225-a0bd-6367e92aa95e admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:18.687794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060853-477b-4225-a0bd-6367e92aa95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:18.688358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060853-477b-4225-a0bd-6367e92aa95e admin admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:18.689862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9060853-477b-4225-a0bd-6367e92aa95e admin admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:18.692348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9060853-477b-4225-a0bd-6367e92aa95e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:18.694394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 144/295] 10.222.0.17 () {56 vars in 1207 bytes} [Tue Oct 18 11:27:18 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:18.700632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd57ab-3047-4c79-9e3f-01a377b99b0a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:18.716892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd57ab-3047-4c79-9e3f-01a377b99b0a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:18.728720 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ded31e-48d3-42b6-a817-21c7e69e61e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:18.729798 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ded31e-48d3-42b6-a817-21c7e69e61e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:18.730318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ded31e-48d3-42b6-a817-21c7e69e61e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:18.730824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ded31e-48d3-42b6-a817-21c7e69e61e5 admin admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/groups/53d323bac77c4ffeab72dc3e5bed383e/roles/04b8e62238cb4198928fd327ef78e6ae` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:18.739012 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ded31e-48d3-42b6-a817-21c7e69e61e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1073f38ce4ec449ba0477f2e9e1786ce, group_id=53d323bac77c4ffeab72dc3e5bed383e, role_id=04b8e62238cb4198928fd327ef78e6ae)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:18.742482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ded31e-48d3-42b6-a817-21c7e69e61e5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:18.764016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 152/296] 10.222.0.17 () {58 vars in 1463 bytes} [Tue Oct 18 11:27:18 2022] PUT /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce/groups/53d323bac77c4ffeab72dc3e5bed383e/roles/04b8e62238cb4198928fd327ef78e6ae => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:19.049051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a01be-5351-4c3b-bcc7-d1eee2aea324 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:19.049399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a01be-5351-4c3b-bcc7-d1eee2aea324 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:19.049585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a01be-5351-4c3b-bcc7-d1eee2aea324 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:19.050884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 145/297] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:19.057680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd1a21-e3d6-4079-90f5-3279ee26262b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:19.057971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd1a21-e3d6-4079-90f5-3279ee26262b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:19.058241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd1a21-e3d6-4079-90f5-3279ee26262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:19.079983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9ffd1a21-e3d6-4079-90f5-3279ee26262b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:19.081255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9ffd1a21-e3d6-4079-90f5-3279ee26262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:19.084600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:19.160098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 153/298] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:19.445727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a01be-5351-4c3b-bcc7-d1eee2aea324 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:19.531943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:19.582784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a01be-5351-4c3b-bcc7-d1eee2aea324 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:19.622208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7e19a-61bb-4723-9bf9-2bc2bf45a6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:19.622857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7e19a-61bb-4723-9bf9-2bc2bf45a6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:19.623133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7e19a-61bb-4723-9bf9-2bc2bf45a6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:19.623404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7e19a-61bb-4723-9bf9-2bc2bf45a6a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:19.635643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7e19a-61bb-4723-9bf9-2bc2bf45a6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-82e7e19a-61bb-4723-9bf9-2bc2bf45a6a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:19.640726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:19.641692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:19.642905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 146/299] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:19.648051 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd1a21-e3d6-4079-90f5-3279ee26262b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:19.774799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd1a21-e3d6-4079-90f5-3279ee26262b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:19.816992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455629a-fdb0-48be-b3e7-8ad1d3096ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:19.817623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455629a-fdb0-48be-b3e7-8ad1d3096ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:19.817902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455629a-fdb0-48be-b3e7-8ad1d3096ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:19.818177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455629a-fdb0-48be-b3e7-8ad1d3096ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:19.818856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455629a-fdb0-48be-b3e7-8ad1d3096ba3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:19.819160 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455629a-fdb0-48be-b3e7-8ad1d3096ba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:19.820558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455629a-fdb0-48be-b3e7-8ad1d3096ba3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:19.830958 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 154/300] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:19.836940 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7e19a-61bb-4723-9bf9-2bc2bf45a6a6 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:19.947337 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7e19a-61bb-4723-9bf9-2bc2bf45a6a6 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:19.967843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:19.993285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c2f5c-6244-4359-9abf-225bf06971e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:19.993911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c2f5c-6244-4359-9abf-225bf06971e9 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:19.994189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c2f5c-6244-4359-9abf-225bf06971e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:19.994457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c2f5c-6244-4359-9abf-225bf06971e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:20.078216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c2f5c-6244-4359-9abf-225bf06971e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:20.079633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c2f5c-6244-4359-9abf-225bf06971e9 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:20.204946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 147/301] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:27:20.210954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455629a-fdb0-48be-b3e7-8ad1d3096ba3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:20.221210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:20.293245 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3eba4-55b0-4315-939d-59bff737ab48 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:20.293568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3eba4-55b0-4315-939d-59bff737ab48 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:20.293804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3eba4-55b0-4315-939d-59bff737ab48 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:20.295454 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 148/302] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:27:20.302373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8979b9-e307-4c03-9404-155042bcc0ef None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:20.302691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8979b9-e307-4c03-9404-155042bcc0ef None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:20.302978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8979b9-e307-4c03-9404-155042bcc0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:20.337631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-cf8979b9-e307-4c03-9404-155042bcc0ef None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:20.338370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cf8979b9-e307-4c03-9404-155042bcc0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:20.356373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:20.389078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455629a-fdb0-48be-b3e7-8ad1d3096ba3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:20.429563 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 149/303] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:20.455241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0830a5-154f-4a94-a03d-857de0bcefa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:20.456132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0830a5-154f-4a94-a03d-857de0bcefa6 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:20.456580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0830a5-154f-4a94-a03d-857de0bcefa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:20.457020 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0830a5-154f-4a94-a03d-857de0bcefa6 None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:20.461575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0830a5-154f-4a94-a03d-857de0bcefa6 None admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:20.463223 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0830a5-154f-4a94-a03d-857de0bcefa6 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:20.465487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 155/304] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:27:20 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:20.470943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8979b9-e307-4c03-9404-155042bcc0ef None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:20.604093 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8979b9-e307-4c03-9404-155042bcc0ef None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:20.608174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beef988-9094-4be6-86f0-76802b00241b None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:20.608781 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beef988-9094-4be6-86f0-76802b00241b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:20.609312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beef988-9094-4be6-86f0-76802b00241b None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:20.616975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 156/305] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:20.626358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a5c4c-4b12-4007-a581-ac560a22629e None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:20.626972 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a5c4c-4b12-4007-a581-ac560a22629e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:20.627447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a5c4c-4b12-4007-a581-ac560a22629e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:20.659580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6734d16-7d4c-4a46-abc7-7a3780b156c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:20.660637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734d16-7d4c-4a46-abc7-7a3780b156c6 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:20.661143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734d16-7d4c-4a46-abc7-7a3780b156c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:20.661614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734d16-7d4c-4a46-abc7-7a3780b156c6 None admin] PATH_INFO: `/v3/groups/53d323bac77c4ffeab72dc3e5bed383e` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:20.662934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6734d16-7d4c-4a46-abc7-7a3780b156c6 None admin] RBAC: Authorizing `identity:get_group(group_id=53d323bac77c4ffeab72dc3e5bed383e)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:20.666453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6734d16-7d4c-4a46-abc7-7a3780b156c6 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:20.672532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 150/306] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Oct 18 11:27:20 2022] GET /identity/v3/groups/53d323bac77c4ffeab72dc3e5bed383e => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:20.673850 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-884a5c4c-4b12-4007-a581-ac560a22629e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:20.675634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-884a5c4c-4b12-4007-a581-ac560a22629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:20.678031 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6734d16-7d4c-4a46-abc7-7a3780b156c6 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:20.705877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:20.789339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:20.792739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 157/307] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:27:20.793710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a5c4c-4b12-4007-a581-ac560a22629e None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:20.838544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6734d16-7d4c-4a46-abc7-7a3780b156c6 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:20.882967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc986354-e138-40b3-9e0e-e4233d93687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:20.883586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc986354-e138-40b3-9e0e-e4233d93687a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:20.883864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc986354-e138-40b3-9e0e-e4233d93687a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:20.884133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc986354-e138-40b3-9e0e-e4233d93687a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:20.884881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc986354-e138-40b3-9e0e-e4233d93687a None admin] RBAC: Adding query filter params (group.id=53d323bac77c4ffeab72dc3e5bed383e, role.id=846f40e28e104ec4bc568fe533f9c9a9, scope.project.id=ba4a0bd34a554747b58ff7dc25238f38) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:20.885193 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc986354-e138-40b3-9e0e-e4233d93687a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:20.886632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc986354-e138-40b3-9e0e-e4233d93687a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:20.900655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 151/308] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Oct 18 11:27:20 2022] GET /identity/v3/role_assignments?group.id=53d323bac77c4ffeab72dc3e5bed383e&scope.project.id=ba4a0bd34a554747b58ff7dc25238f38&role.id=846f40e28e104ec4bc568fe533f9c9a9 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:20.995816 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a5c4c-4b12-4007-a581-ac560a22629e None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:21.046060 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e9398-515f-4d5e-80be-2e9e1f15398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:21.047066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e9398-515f-4d5e-80be-2e9e1f15398a None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:21.047573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e9398-515f-4d5e-80be-2e9e1f15398a None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:21.048061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e9398-515f-4d5e-80be-2e9e1f15398a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:21.065156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e9398-515f-4d5e-80be-2e9e1f15398a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-625e9398-515f-4d5e-80be-2e9e1f15398a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:21.070586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:21.071528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 18 11:27:21.072303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:21.074753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 158/309] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:21.081494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc986354-e138-40b3-9e0e-e4233d93687a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:21.154600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e9398-515f-4d5e-80be-2e9e1f15398a None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:21.208870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:21.235349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc986354-e138-40b3-9e0e-e4233d93687a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:21.243754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:21.278171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042030b3-f43b-4179-8c98-94bb2d941c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:21.278795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042030b3-f43b-4179-8c98-94bb2d941c3f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:21.279074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042030b3-f43b-4179-8c98-94bb2d941c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:21.279345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042030b3-f43b-4179-8c98-94bb2d941c3f None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:21.280561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042030b3-f43b-4179-8c98-94bb2d941c3f None admin] RBAC: Adding query filter params (name=member) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:21.281559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042030b3-f43b-4179-8c98-94bb2d941c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:21.283287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042030b3-f43b-4179-8c98-94bb2d941c3f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:21.294004 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 152/310] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:27:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:21.295111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e9398-515f-4d5e-80be-2e9e1f15398a None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:21.301720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042030b3-f43b-4179-8c98-94bb2d941c3f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:21.343974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ec9d2-5561-4445-9f53-52315d018cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:21.344684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ec9d2-5561-4445-9f53-52315d018cbd None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:21.345280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ec9d2-5561-4445-9f53-52315d018cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:21.345557 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ec9d2-5561-4445-9f53-52315d018cbd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:21.364578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ec9d2-5561-4445-9f53-52315d018cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-cb2ec9d2-5561-4445-9f53-52315d018cbd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:21.371168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:21.372152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:27:21.373019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:21.377962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 159/311] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:27:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:21.385634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ec9d2-5561-4445-9f53-52315d018cbd None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:21.440672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042030b3-f43b-4179-8c98-94bb2d941c3f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:21.481275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2061b-c26c-4557-8c0e-3be70b108a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:21.483685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2061b-c26c-4557-8c0e-3be70b108a9a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:21.484175 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2061b-c26c-4557-8c0e-3be70b108a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:21.484653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2061b-c26c-4557-8c0e-3be70b108a9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:21.498265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:21.511241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ec9d2-5561-4445-9f53-52315d018cbd None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:21.535415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:21.559597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284de043-b5bf-473a-b9ed-f064a5262ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:21.561572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284de043-b5bf-473a-b9ed-f064a5262ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:21.561961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284de043-b5bf-473a-b9ed-f064a5262ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:21.562333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284de043-b5bf-473a-b9ed-f064a5262ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:21.563143 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284de043-b5bf-473a-b9ed-f064a5262ad1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:21.563554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284de043-b5bf-473a-b9ed-f064a5262ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:21.565121 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284de043-b5bf-473a-b9ed-f064a5262ad1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:21.574630 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2061b-c26c-4557-8c0e-3be70b108a9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:21.575230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 160/312] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:21.576603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2061b-c26c-4557-8c0e-3be70b108a9a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:21.584833 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284de043-b5bf-473a-b9ed-f064a5262ad1 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:21.718352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 153/313] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 18 11:27:21.721102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284de043-b5bf-473a-b9ed-f064a5262ad1 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:21.726252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2061b-c26c-4557-8c0e-3be70b108a9a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:21.763996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812c7a5-9d06-43e1-8b19-c5b3d1004236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:21.764658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812c7a5-9d06-43e1-8b19-c5b3d1004236 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:21.765013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812c7a5-9d06-43e1-8b19-c5b3d1004236 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:21.765292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812c7a5-9d06-43e1-8b19-c5b3d1004236 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:21.777557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:21.829379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:21.852165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812c7a5-9d06-43e1-8b19-c5b3d1004236 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:21.854123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812c7a5-9d06-43e1-8b19-c5b3d1004236 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:21.859207 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2061b-c26c-4557-8c0e-3be70b108a9a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:21.915371 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b773f7-24e9-45cb-9f59-867e2377bf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:21.916026 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b773f7-24e9-45cb-9f59-867e2377bf59 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:21.916325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b773f7-24e9-45cb-9f59-867e2377bf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:21.916579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b773f7-24e9-45cb-9f59-867e2377bf59 None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:21.917790 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b773f7-24e9-45cb-9f59-867e2377bf59 None admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:21.919262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b773f7-24e9-45cb-9f59-867e2377bf59 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:21.921756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 154/314] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:27:21 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:21.922828 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1dbc-bd79-470d-87b5-adf2409c3af3 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:21.923112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1dbc-bd79-470d-87b5-adf2409c3af3 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:21.923384 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1dbc-bd79-470d-87b5-adf2409c3af3 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:21.925186 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 155/315] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:21.925905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca3039-7a78-4326-aafe-4217aa0d6c83 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:21.926010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca3039-7a78-4326-aafe-4217aa0d6c83 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:21.926276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca3039-7a78-4326-aafe-4217aa0d6c83 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:21.927522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 156/316] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:21.928245 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca3039-7a78-4326-aafe-4217aa0d6c83 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:21.987168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 161/317] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:27:21.988246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084bde2-673f-49b9-8140-2e71f586fa0a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:21.988532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084bde2-673f-49b9-8140-2e71f586fa0a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:21.988806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084bde2-673f-49b9-8140-2e71f586fa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:22.008754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7084bde2-673f-49b9-8140-2e71f586fa0a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:22.010497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7084bde2-673f-49b9-8140-2e71f586fa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:22.062950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca3039-7a78-4326-aafe-4217aa0d6c83 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:22.074955 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:22.084290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 162/318] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:22.085649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef24dec-f0da-4ae5-972b-c006f1dd68a1 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:22.085936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef24dec-f0da-4ae5-972b-c006f1dd68a1 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:22.086209 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef24dec-f0da-4ae5-972b-c006f1dd68a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:22.105765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887d276-3fc5-44db-85b3-b742b8b582bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:22.106409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887d276-3fc5-44db-85b3-b742b8b582bb None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:22.106686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887d276-3fc5-44db-85b3-b742b8b582bb None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:22.106957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887d276-3fc5-44db-85b3-b742b8b582bb None admin] PATH_INFO: `/v3/groups/53d323bac77c4ffeab72dc3e5bed383e` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:22.108642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887d276-3fc5-44db-85b3-b742b8b582bb None admin] RBAC: Authorizing `identity:get_group(group_id=53d323bac77c4ffeab72dc3e5bed383e)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:22.109860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4ef24dec-f0da-4ae5-972b-c006f1dd68a1 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:22.110307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887d276-3fc5-44db-85b3-b742b8b582bb None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:22.111055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4ef24dec-f0da-4ae5-972b-c006f1dd68a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:22.111899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 157/319] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Oct 18 11:27:21 2022] GET /identity/v3/groups/53d323bac77c4ffeab72dc3e5bed383e => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:22.112837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887d276-3fc5-44db-85b3-b742b8b582bb None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:22.122426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:22.205566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 163/320] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:22.206322 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef24dec-f0da-4ae5-972b-c006f1dd68a1 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:22.248872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887d276-3fc5-44db-85b3-b742b8b582bb None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:22.289267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31850c9e-eb2c-46a2-9218-a69521638d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:22.290288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31850c9e-eb2c-46a2-9218-a69521638d55 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:22.290762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31850c9e-eb2c-46a2-9218-a69521638d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:22.291332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31850c9e-eb2c-46a2-9218-a69521638d55 None admin] PATH_INFO: `/v3/projects/ba4a0bd34a554747b58ff7dc25238f38` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:22.292968 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31850c9e-eb2c-46a2-9218-a69521638d55 None admin] RBAC: Authorizing `identity:get_project(project_id=ba4a0bd34a554747b58ff7dc25238f38)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:22.294776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31850c9e-eb2c-46a2-9218-a69521638d55 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:22.297496 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 158/321] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:27:22 2022] GET /identity/v3/projects/ba4a0bd34a554747b58ff7dc25238f38 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:22.305862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31850c9e-eb2c-46a2-9218-a69521638d55 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:22.345955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef24dec-f0da-4ae5-972b-c006f1dd68a1 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:22.363260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:22.394663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc6c55-107a-4afd-ba3d-d01ba1c7cda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:22.395294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc6c55-107a-4afd-ba3d-d01ba1c7cda1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:22.395636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc6c55-107a-4afd-ba3d-d01ba1c7cda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:22.395913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc6c55-107a-4afd-ba3d-d01ba1c7cda1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:22.396699 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc6c55-107a-4afd-ba3d-d01ba1c7cda1 None admin] RBAC: Adding query filter params (group.id=53d323bac77c4ffeab72dc3e5bed383e, role.id=846f40e28e104ec4bc568fe533f9c9a9, scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:22.397033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc6c55-107a-4afd-ba3d-d01ba1c7cda1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:22.398490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc6c55-107a-4afd-ba3d-d01ba1c7cda1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:22.407999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 164/322] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Oct 18 11:27:22 2022] GET /identity/v3/role_assignments?group.id=53d323bac77c4ffeab72dc3e5bed383e&scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce&role.id=846f40e28e104ec4bc568fe533f9c9a9 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:22.417786 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc6c55-107a-4afd-ba3d-d01ba1c7cda1 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:22.428555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:22.444646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31850c9e-eb2c-46a2-9218-a69521638d55 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:22.487803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7f819-4b5a-43da-8cb9-afe6775d3a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:22.489647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7f819-4b5a-43da-8cb9-afe6775d3a65 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:22.490133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7f819-4b5a-43da-8cb9-afe6775d3a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:22.490623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7f819-4b5a-43da-8cb9-afe6775d3a65 None admin] PATH_INFO: `/v3/groups/53d323bac77c4ffeab72dc3e5bed383e` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:22.492095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7f819-4b5a-43da-8cb9-afe6775d3a65 None admin] RBAC: Authorizing `identity:get_group(group_id=53d323bac77c4ffeab72dc3e5bed383e)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:22.494089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7f819-4b5a-43da-8cb9-afe6775d3a65 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:22.496451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 159/323] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Oct 18 11:27:22 2022] GET /identity/v3/groups/53d323bac77c4ffeab72dc3e5bed383e => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:22.502334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7f819-4b5a-43da-8cb9-afe6775d3a65 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:22.565100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc6c55-107a-4afd-ba3d-d01ba1c7cda1 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:22.611794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2710b6d-df54-4653-aa7c-d4c2f974ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:22.612444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2710b6d-df54-4653-aa7c-d4c2f974ad22 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:22.612720 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2710b6d-df54-4653-aa7c-d4c2f974ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:22.613041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2710b6d-df54-4653-aa7c-d4c2f974ad22 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:22.626019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2710b6d-df54-4653-aa7c-d4c2f974ad22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-c2710b6d-df54-4653-aa7c-d4c2f974ad22 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:22.631548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:22.632415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:22.633246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 165/324] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:27:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:22.633893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2710b6d-df54-4653-aa7c-d4c2f974ad22 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:22.661499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7f819-4b5a-43da-8cb9-afe6775d3a65 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:22.679627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:22.709504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847b9e3-0e28-4fa1-a0e3-1d7590c76d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:22.710313 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847b9e3-0e28-4fa1-a0e3-1d7590c76d7f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:22.710593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847b9e3-0e28-4fa1-a0e3-1d7590c76d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:22.710867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847b9e3-0e28-4fa1-a0e3-1d7590c76d7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:22.712094 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847b9e3-0e28-4fa1-a0e3-1d7590c76d7f None admin] RBAC: Adding query filter params (group.id=53d323bac77c4ffeab72dc3e5bed383e, role.id=04b8e62238cb4198928fd327ef78e6ae, scope.project.id=ba4a0bd34a554747b58ff7dc25238f38) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:22.712420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847b9e3-0e28-4fa1-a0e3-1d7590c76d7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:22.714250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847b9e3-0e28-4fa1-a0e3-1d7590c76d7f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:22.715449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:22.723030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 160/325] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Oct 18 11:27:22 2022] GET /identity/v3/role_assignments?group.id=53d323bac77c4ffeab72dc3e5bed383e&scope.project.id=ba4a0bd34a554747b58ff7dc25238f38&role.id=04b8e62238cb4198928fd327ef78e6ae => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:22.723863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847b9e3-0e28-4fa1-a0e3-1d7590c76d7f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:22.771049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2710b6d-df54-4653-aa7c-d4c2f974ad22 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:22.828488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1417c7-1be0-49e4-ae81-c4560c8228f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:22.829248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1417c7-1be0-49e4-ae81-c4560c8228f5 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:22.829526 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1417c7-1be0-49e4-ae81-c4560c8228f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:22.829796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1417c7-1be0-49e4-ae81-c4560c8228f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:22.845144 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1417c7-1be0-49e4-ae81-c4560c8228f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-1a1417c7-1be0-49e4-ae81-c4560c8228f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:22.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:22.852908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:22.853723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 166/326] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:27:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:22.859366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1417c7-1be0-49e4-ae81-c4560c8228f5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:22.883263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847b9e3-0e28-4fa1-a0e3-1d7590c76d7f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:22.934818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59219246-822d-4203-9060-a2d341b839bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:22.935432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59219246-822d-4203-9060-a2d341b839bf None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:22.935707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59219246-822d-4203-9060-a2d341b839bf None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:22.936539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59219246-822d-4203-9060-a2d341b839bf None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:22.937241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59219246-822d-4203-9060-a2d341b839bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:22.937541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59219246-822d-4203-9060-a2d341b839bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:22.938904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59219246-822d-4203-9060-a2d341b839bf None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:22.947900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 161/327] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:22.955137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59219246-822d-4203-9060-a2d341b839bf None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:22.999144 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1417c7-1be0-49e4-ae81-c4560c8228f5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:23.000349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:23.030219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:23.042283 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273d27f-b0f0-4081-96bd-a14095a493c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:23.042904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273d27f-b0f0-4081-96bd-a14095a493c5 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:23.043182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273d27f-b0f0-4081-96bd-a14095a493c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:23.043453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273d27f-b0f0-4081-96bd-a14095a493c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:23.044127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273d27f-b0f0-4081-96bd-a14095a493c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:23.044425 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273d27f-b0f0-4081-96bd-a14095a493c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:23.046848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273d27f-b0f0-4081-96bd-a14095a493c5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:23.056295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 167/328] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:23.062610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273d27f-b0f0-4081-96bd-a14095a493c5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:23.074710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59219246-822d-4203-9060-a2d341b839bf None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:23.115867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5f5e4-b62d-4312-b86a-e22426e05db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:23.116521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5f5e4-b62d-4312-b86a-e22426e05db7 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:23.116822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5f5e4-b62d-4312-b86a-e22426e05db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:23.117158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5f5e4-b62d-4312-b86a-e22426e05db7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:23.191966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273d27f-b0f0-4081-96bd-a14095a493c5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:23.205852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5f5e4-b62d-4312-b86a-e22426e05db7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:23.207318 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5f5e4-b62d-4312-b86a-e22426e05db7 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:23.232232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8514e0e-78f4-4046-ba10-81077b0a4d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:23.233456 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8514e0e-78f4-4046-ba10-81077b0a4d78 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:23.234370 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8514e0e-78f4-4046-ba10-81077b0a4d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:23.234659 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8514e0e-78f4-4046-ba10-81077b0a4d78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:23.262088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:23.300188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:23.314513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8514e0e-78f4-4046-ba10-81077b0a4d78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:23.316453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8514e0e-78f4-4046-ba10-81077b0a4d78 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:23.342219 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 162/329] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 18 11:27:23.347886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5f5e4-b62d-4312-b86a-e22426e05db7 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:23.423621 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 168/330] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 18 11:27:23.434189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8514e0e-78f4-4046-ba10-81077b0a4d78 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:23.454920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5f5e4-b62d-4312-b86a-e22426e05db7 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:23.490978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce6567-f96e-4ffa-b5d7-79410a661e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:23.491622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce6567-f96e-4ffa-b5d7-79410a661e6c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:23.491869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce6567-f96e-4ffa-b5d7-79410a661e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:23.492133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce6567-f96e-4ffa-b5d7-79410a661e6c None admin] PATH_INFO: `/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:23.493278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce6567-f96e-4ffa-b5d7-79410a661e6c None admin] RBAC: Authorizing `identity:get_project(project_id=1073f38ce4ec449ba0477f2e9e1786ce)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:23.494712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce6567-f96e-4ffa-b5d7-79410a661e6c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:23.496330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 163/331] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:27:23 2022] GET /identity/v3/projects/1073f38ce4ec449ba0477f2e9e1786ce => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:23.501864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce6567-f96e-4ffa-b5d7-79410a661e6c None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:23.510275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:23.524065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:23.536980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8514e0e-78f4-4046-ba10-81077b0a4d78 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:23.572035 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233c57b-919b-4e1c-9f62-609b5e1f04f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:23.572599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233c57b-919b-4e1c-9f62-609b5e1f04f9 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:23.572897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233c57b-919b-4e1c-9f62-609b5e1f04f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:23.573175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233c57b-919b-4e1c-9f62-609b5e1f04f9 None admin] PATH_INFO: `/v3/projects/4c023e59351c4a06998c34820bba9111` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:23.574328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233c57b-919b-4e1c-9f62-609b5e1f04f9 None admin] RBAC: Authorizing `identity:get_project(project_id=4c023e59351c4a06998c34820bba9111)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:23.575764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233c57b-919b-4e1c-9f62-609b5e1f04f9 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:23.577843 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 169/332] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:27:23 2022] GET /identity/v3/projects/4c023e59351c4a06998c34820bba9111 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:23.583245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233c57b-919b-4e1c-9f62-609b5e1f04f9 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:23.602710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce6567-f96e-4ffa-b5d7-79410a661e6c None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:23.636501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d225f74-27bb-4e4a-83d9-7621354d2eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:23.637170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d225f74-27bb-4e4a-83d9-7621354d2eea None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:23.637449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d225f74-27bb-4e4a-83d9-7621354d2eea None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:23.637722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d225f74-27bb-4e4a-83d9-7621354d2eea None admin] PATH_INFO: `/v3/groups/53d323bac77c4ffeab72dc3e5bed383e` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:23.638944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d225f74-27bb-4e4a-83d9-7621354d2eea None admin] RBAC: Authorizing `identity:get_group(group_id=53d323bac77c4ffeab72dc3e5bed383e)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:23.640424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d225f74-27bb-4e4a-83d9-7621354d2eea None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:23.642028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 164/333] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Oct 18 11:27:23 2022] GET /identity/v3/groups/53d323bac77c4ffeab72dc3e5bed383e => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:23.647778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d225f74-27bb-4e4a-83d9-7621354d2eea None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:23.683281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233c57b-919b-4e1c-9f62-609b5e1f04f9 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:23.715744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8f806-4505-468f-b357-410777792e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:23.716381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8f806-4505-468f-b357-410777792e0d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:23.716660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8f806-4505-468f-b357-410777792e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:23.716961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8f806-4505-468f-b357-410777792e0d None admin] PATH_INFO: `/v3/groups/52d3ec9bcb0641df8c8f39185bbca677` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:23.718603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8f806-4505-468f-b357-410777792e0d None admin] RBAC: Authorizing `identity:get_group(group_id=52d3ec9bcb0641df8c8f39185bbca677)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:23.720051 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8f806-4505-468f-b357-410777792e0d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:23.721669 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 170/334] 10.222.0.17 () {56 vars in 1180 bytes} [Tue Oct 18 11:27:23 2022] GET /identity/v3/groups/52d3ec9bcb0641df8c8f39185bbca677 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:23.722023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:23.727639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8f806-4505-468f-b357-410777792e0d None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:23.748411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d225f74-27bb-4e4a-83d9-7621354d2eea None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:23.752924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:23.783358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfa2d1-ac86-4620-841f-f9c2ce64d086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:23.783971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfa2d1-ac86-4620-841f-f9c2ce64d086 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:23.784544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfa2d1-ac86-4620-841f-f9c2ce64d086 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:23.784544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfa2d1-ac86-4620-841f-f9c2ce64d086 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:23.785255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfa2d1-ac86-4620-841f-f9c2ce64d086 None admin] RBAC: Adding query filter params (group.id=53d323bac77c4ffeab72dc3e5bed383e, role.id=04b8e62238cb4198928fd327ef78e6ae, scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:23.785567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfa2d1-ac86-4620-841f-f9c2ce64d086 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:23.786998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfa2d1-ac86-4620-841f-f9c2ce64d086 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:23.804874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 165/335] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Oct 18 11:27:23 2022] GET /identity/v3/role_assignments?group.id=53d323bac77c4ffeab72dc3e5bed383e&scope.project.id=1073f38ce4ec449ba0477f2e9e1786ce&role.id=04b8e62238cb4198928fd327ef78e6ae => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:23.836016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8f806-4505-468f-b357-410777792e0d None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:23.872575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6fd72-3b05-418b-a457-6a9138f27153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:23.873228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6fd72-3b05-418b-a457-6a9138f27153 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:23.873540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6fd72-3b05-418b-a457-6a9138f27153 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:23.873811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6fd72-3b05-418b-a457-6a9138f27153 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:23.874549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6fd72-3b05-418b-a457-6a9138f27153 None admin] RBAC: Adding query filter params (group.id=52d3ec9bcb0641df8c8f39185bbca677, role.id=8d9ff9da6ad946a6bc49336777182ff9, scope.project.id=4c023e59351c4a06998c34820bba9111) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:23.874855 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6fd72-3b05-418b-a457-6a9138f27153 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:23.876311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6fd72-3b05-418b-a457-6a9138f27153 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:23.882777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 171/336] 10.222.0.17 () {56 vars in 1376 bytes} [Tue Oct 18 11:27:23 2022] GET /identity/v3/role_assignments?group.id=52d3ec9bcb0641df8c8f39185bbca677&scope.project.id=4c023e59351c4a06998c34820bba9111&role.id=8d9ff9da6ad946a6bc49336777182ff9 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:26.568374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce22659-15f5-480c-9004-1483a3786738 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:26.568961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce22659-15f5-480c-9004-1483a3786738 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:26.568961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce22659-15f5-480c-9004-1483a3786738 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:26.570457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 166/337] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:26.586768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c35f53-b977-4569-95fa-53b6aecfe4a5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:26.587717 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c35f53-b977-4569-95fa-53b6aecfe4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:26.588196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c35f53-b977-4569-95fa-53b6aecfe4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:26.612337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-72c35f53-b977-4569-95fa-53b6aecfe4a5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:26.613965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-72c35f53-b977-4569-95fa-53b6aecfe4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:26.710132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 172/338] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:26.746743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689684f-83f4-4888-b15c-20a2ba623fae None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:26.747374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689684f-83f4-4888-b15c-20a2ba623fae None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:26.747862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689684f-83f4-4888-b15c-20a2ba623fae None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:26.749443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 167/339] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:26.754888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b06ba-28d4-4abd-9498-93351a9506a3 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:26.755413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b06ba-28d4-4abd-9498-93351a9506a3 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:26.755888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b06ba-28d4-4abd-9498-93351a9506a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:26.777837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4f5b06ba-28d4-4abd-9498-93351a9506a3 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:26.779348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4f5b06ba-28d4-4abd-9498-93351a9506a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:26.799234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:26.883259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 173/340] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:27.056973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e89f0b-4d7b-42e3-8f28-1e40cfc53ef9 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:27.057331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e89f0b-4d7b-42e3-8f28-1e40cfc53ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:27.057526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e89f0b-4d7b-42e3-8f28-1e40cfc53ef9 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:27.059039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 168/341] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:27.064682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756ec8f-1d09-4ce2-a264-147c235e194c None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:27.065000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756ec8f-1d09-4ce2-a264-147c235e194c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:27.065272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756ec8f-1d09-4ce2-a264-147c235e194c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:27.089098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3756ec8f-1d09-4ce2-a264-147c235e194c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:27.090287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3756ec8f-1d09-4ce2-a264-147c235e194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:27.110624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e89f0b-4d7b-42e3-8f28-1e40cfc53ef9 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:27.193874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 174/342] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:27.269516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e89f0b-4d7b-42e3-8f28-1e40cfc53ef9 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:27.312675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089e7a6-37ba-45ff-ba5e-41855f9e4d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:27.313341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e7a6-37ba-45ff-ba5e-41855f9e4d20 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:27.313620 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e7a6-37ba-45ff-ba5e-41855f9e4d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:27.313894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e7a6-37ba-45ff-ba5e-41855f9e4d20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:27.326398 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:27.331044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089e7a6-37ba-45ff-ba5e-41855f9e4d20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:27.332581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089e7a6-37ba-45ff-ba5e-41855f9e4d20 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:27.346689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756ec8f-1d09-4ce2-a264-147c235e194c None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-0089e7a6-37ba-45ff-ba5e-41855f9e4d20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:27.363634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:27.364964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:27.365856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:27.365856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:27.365856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:27.367009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 169/343] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:27.371175 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089e7a6-37ba-45ff-ba5e-41855f9e4d20 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:27.493485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:27.520047 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756ec8f-1d09-4ce2-a264-147c235e194c None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:27.540745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089e7a6-37ba-45ff-ba5e-41855f9e4d20 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:27.557677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503529b2-10ad-4297-bc72-c6f8fbe99270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:27.558432 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503529b2-10ad-4297-bc72-c6f8fbe99270 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:27.558708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503529b2-10ad-4297-bc72-c6f8fbe99270 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:27.558974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503529b2-10ad-4297-bc72-c6f8fbe99270 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:27.569954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503529b2-10ad-4297-bc72-c6f8fbe99270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:27.571405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503529b2-10ad-4297-bc72-c6f8fbe99270 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:27.575753 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf8a0b-db66-4c49-b6d6-8b9fb65f6376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:27.576365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf8a0b-db66-4c49-b6d6-8b9fb65f6376 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:27.577252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf8a0b-db66-4c49-b6d6-8b9fb65f6376 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:27.577527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf8a0b-db66-4c49-b6d6-8b9fb65f6376 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:27.578170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf8a0b-db66-4c49-b6d6-8b9fb65f6376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:27.578469 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf8a0b-db66-4c49-b6d6-8b9fb65f6376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:27.580624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf8a0b-db66-4c49-b6d6-8b9fb65f6376 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-503529b2-10ad-4297-bc72-c6f8fbe99270 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:27.587585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:27.588921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:27.589784 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:27.589784 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:27.589784 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:27.589784 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 175/344] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:27.592777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db679c-b975-4dff-b97b-a56485098a6f None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:27.593870 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db679c-b975-4dff-b97b-a56485098a6f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:27.594380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db679c-b975-4dff-b97b-a56485098a6f None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:27.596489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 176/345] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:27.600869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 170/346] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:27.600869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf8a0b-db66-4c49-b6d6-8b9fb65f6376 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:27.601269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db679c-b975-4dff-b97b-a56485098a6f None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:27.655711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:27.706487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf8a0b-db66-4c49-b6d6-8b9fb65f6376 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:27.714458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db679c-b975-4dff-b97b-a56485098a6f None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:27.741926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a364f52-2687-40e2-8171-7822f517c64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:27.742543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a364f52-2687-40e2-8171-7822f517c64e None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:27.742820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a364f52-2687-40e2-8171-7822f517c64e None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:27.743091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a364f52-2687-40e2-8171-7822f517c64e None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:27.743826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a364f52-2687-40e2-8171-7822f517c64e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:27.744034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a364f52-2687-40e2-8171-7822f517c64e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:27.745441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a364f52-2687-40e2-8171-7822f517c64e None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:27.745901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:27.752996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da568-b251-4f7c-876b-5de598382e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:27.754028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da568-b251-4f7c-876b-5de598382e6b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:27.754308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da568-b251-4f7c-876b-5de598382e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:27.754575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da568-b251-4f7c-876b-5de598382e6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:27.757330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 171/347] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:27.762779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c35b8-eab2-4e63-a3c1-e5b9a363d055 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:27.763066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c35b8-eab2-4e63-a3c1-e5b9a363d055 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:27.763342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c35b8-eab2-4e63-a3c1-e5b9a363d055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:27.766291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da568-b251-4f7c-876b-5de598382e6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:27.767956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da568-b251-4f7c-876b-5de598382e6b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:27.782044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-138c35b8-eab2-4e63-a3c1-e5b9a363d055 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:27.783260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-138c35b8-eab2-4e63-a3c1-e5b9a363d055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-ec5da568-b251-4f7c-876b-5de598382e6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:27.783689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:27.784555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:27.789908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:27.789908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:27.789908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:27.789908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 177/348] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:27.789908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da568-b251-4f7c-876b-5de598382e6b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:27.857341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 172/349] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:27.858452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c35b8-eab2-4e63-a3c1-e5b9a363d055 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:27.906634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da568-b251-4f7c-876b-5de598382e6b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:27.912580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:27.953435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ae7b-0fd4-4927-b13a-42e6a5479a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:27.954473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ae7b-0fd4-4927-b13a-42e6a5479a86 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:27.954751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ae7b-0fd4-4927-b13a-42e6a5479a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:27.955022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ae7b-0fd4-4927-b13a-42e6a5479a86 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:27.955980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ae7b-0fd4-4927-b13a-42e6a5479a86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:27.957772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ae7b-0fd4-4927-b13a-42e6a5479a86 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:27.980761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c35b8-eab2-4e63-a3c1-e5b9a363d055 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:27.993421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 178/350] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:27:27 2022] POST /identity/v3/users => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:27:28.000209 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ae7b-0fd4-4927-b13a-42e6a5479a86 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:28.052849 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48fbf4-9dd7-44e2-92b2-4a6663b08003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:28.053536 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48fbf4-9dd7-44e2-92b2-4a6663b08003 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:28.053818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48fbf4-9dd7-44e2-92b2-4a6663b08003 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:28.054085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48fbf4-9dd7-44e2-92b2-4a6663b08003 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:28.055690 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48fbf4-9dd7-44e2-92b2-4a6663b08003 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:28.058994 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48fbf4-9dd7-44e2-92b2-4a6663b08003 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:28.086316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:28.101924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 173/351] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:27:27 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:27:28.103396 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb011d-78a2-48e6-a9d9-634eac06e6c8 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:28.103684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb011d-78a2-48e6-a9d9-634eac06e6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:28.103955 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb011d-78a2-48e6-a9d9-634eac06e6c8 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:28.105672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 174/352] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:28.111492 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd80cc-f65f-41d2-86d7-e9af8caf153b None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:28.111764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd80cc-f65f-41d2-86d7-e9af8caf153b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:28.112032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd80cc-f65f-41d2-86d7-e9af8caf153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:28.132900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-b5dd80cc-f65f-41d2-86d7-e9af8caf153b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:28.134692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b5dd80cc-f65f-41d2-86d7-e9af8caf153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:28.190746 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ae7b-0fd4-4927-b13a-42e6a5479a86 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:28.234953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 175/353] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:28.235970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd80cc-f65f-41d2-86d7-e9af8caf153b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:28.273607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b117a-faf9-4aed-9608-da296aadae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:28.274603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:28.275309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b117a-faf9-4aed-9608-da296aadae4e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:28.275619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b117a-faf9-4aed-9608-da296aadae4e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:28.275924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b117a-faf9-4aed-9608-da296aadae4e None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:28.277137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b117a-faf9-4aed-9608-da296aadae4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:28.277571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b117a-faf9-4aed-9608-da296aadae4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:28.279334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b117a-faf9-4aed-9608-da296aadae4e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:28.296098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 179/354] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:28.301828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b117a-faf9-4aed-9608-da296aadae4e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:28.378231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd80cc-f65f-41d2-86d7-e9af8caf153b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:28.426390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eede2-2115-4a28-8104-fd5fd761de7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:28.427581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eede2-2115-4a28-8104-fd5fd761de7a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:28.427962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eede2-2115-4a28-8104-fd5fd761de7a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:28.428339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eede2-2115-4a28-8104-fd5fd761de7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:28.441964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:28.444239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eede2-2115-4a28-8104-fd5fd761de7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:28.445599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b117a-faf9-4aed-9608-da296aadae4e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:28.445975 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eede2-2115-4a28-8104-fd5fd761de7a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-bc8eede2-2115-4a28-8104-fd5fd761de7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:28.467134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:28.468270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:28.469299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:28.469299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:28.469299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:28.470677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 176/355] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:28.478334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eede2-2115-4a28-8104-fd5fd761de7a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:28.492968 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ab247-4a87-49d7-8603-2830284587bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:28.493826 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ab247-4a87-49d7-8603-2830284587bb None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:28.494260 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ab247-4a87-49d7-8603-2830284587bb None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:28.495828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ab247-4a87-49d7-8603-2830284587bb None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:28.496715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ab247-4a87-49d7-8603-2830284587bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:28.498917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ab247-4a87-49d7-8603-2830284587bb None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:28.529196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 180/356] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:27:28 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:27:28.575521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:28.579793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ab247-4a87-49d7-8603-2830284587bb None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:28.606200 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eede2-2115-4a28-8104-fd5fd761de7a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:28.651733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ef2d4-df7d-433e-bb97-a2c93aedfbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:28.652360 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ef2d4-df7d-433e-bb97-a2c93aedfbac None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:28.652644 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ef2d4-df7d-433e-bb97-a2c93aedfbac None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:28.652958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ef2d4-df7d-433e-bb97-a2c93aedfbac None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:28.653616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ef2d4-df7d-433e-bb97-a2c93aedfbac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:28.653921 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ef2d4-df7d-433e-bb97-a2c93aedfbac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:28.657078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ef2d4-df7d-433e-bb97-a2c93aedfbac None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:28.676428 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 177/357] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:28.677310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ef2d4-df7d-433e-bb97-a2c93aedfbac None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:28.783959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ab247-4a87-49d7-8603-2830284587bb None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:28.830595 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d4100-fa09-4691-a286-86db6a063af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:28.831625 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d4100-fa09-4691-a286-86db6a063af7 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:28.832124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d4100-fa09-4691-a286-86db6a063af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:28.832596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d4100-fa09-4691-a286-86db6a063af7 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:28.833506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d4100-fa09-4691-a286-86db6a063af7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:28.835231 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d4100-fa09-4691-a286-86db6a063af7 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:28.844986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.sql.core [None req-b11d4100-fa09-4691-a286-86db6a063af7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 18 11:27:28.844986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 18 11:27:28.844986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [parameters: {'id': '12fc916e7bed44209a52b03c97bd75e5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 18 11:27:28.844986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81701) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 18 11:27:28.854869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ef2d4-df7d-433e-bb97-a2c93aedfbac None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-b11d4100-fa09-4691-a286-86db6a063af7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result = self._query(query) Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application conn.query(q) Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result.read() Oct 18 11:27:28.863309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return ref.to_dict() Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 18 11:27:28.864535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application next(self.gen) Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application yield resource Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application next(self.gen) Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.session.rollback() Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.force_reraise() Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise self.value Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application session.commit() Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 18 11:27:28.865478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._prepare_impl() Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.session.flush() Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._flush(objects) Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application compat.raise_( Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application flush_context.execute() Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rec.execute(self) Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 18 11:27:28.866306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 18 11:27:28.867174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result = self._query(query) Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application conn.query(q) Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result.read() Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 18 11:27:28.867987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [parameters: {'id': '12fc916e7bed44209a52b03c97bd75e5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:28.870427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 18 11:27:28.872453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:28.876417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 181/358] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Oct 18 11:27:28 2022] POST /identity/v3/roles => generated 144 bytes in 297 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:28.882815 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d4100-fa09-4691-a286-86db6a063af7 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:28.901462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb91a4e-dec4-4515-8814-9d7919038b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:28.902146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb91a4e-dec4-4515-8814-9d7919038b7b None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:28.902470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb91a4e-dec4-4515-8814-9d7919038b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:28.902754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb91a4e-dec4-4515-8814-9d7919038b7b None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:28.903462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb91a4e-dec4-4515-8814-9d7919038b7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:28.905956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb91a4e-dec4-4515-8814-9d7919038b7b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:28.909603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:28.918983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:28.935320 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 178/359] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:27:28 2022] POST /identity/v3/users => generated 249 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:27:29.020489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d4100-fa09-4691-a286-86db6a063af7 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:29.062764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3be5de-2586-49c4-bc0d-6bcc370bf12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:29.063656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3be5de-2586-49c4-bc0d-6bcc370bf12b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:29.064076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3be5de-2586-49c4-bc0d-6bcc370bf12b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:29.064499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3be5de-2586-49c4-bc0d-6bcc370bf12b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:29.077758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3be5de-2586-49c4-bc0d-6bcc370bf12b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-5a3be5de-2586-49c4-bc0d-6bcc370bf12b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:29.083167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:29.084289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 18 11:27:29.085296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:29.087706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 182/360] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:27:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:29.094586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb91a4e-dec4-4515-8814-9d7919038b7b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:29.217045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb91a4e-dec4-4515-8814-9d7919038b7b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:29.252058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d60e3c-93ea-42b0-8953-d96ab1d457db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:29.252730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d60e3c-93ea-42b0-8953-d96ab1d457db None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:29.253045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d60e3c-93ea-42b0-8953-d96ab1d457db None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:29.253323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d60e3c-93ea-42b0-8953-d96ab1d457db None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:29.254009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d60e3c-93ea-42b0-8953-d96ab1d457db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:29.254314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d60e3c-93ea-42b0-8953-d96ab1d457db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:29.255700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d60e3c-93ea-42b0-8953-d96ab1d457db None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:29.265067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 179/361] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:30.791458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bc7e3-d62b-4505-b5ce-a0d587e701d9 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:30.792627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bc7e3-d62b-4505-b5ce-a0d587e701d9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:30.793144 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bc7e3-d62b-4505-b5ce-a0d587e701d9 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:30.794936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 183/362] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:30.801378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7bc51-1327-41f6-92a2-0b642d84e780 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:30.802264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7bc51-1327-41f6-92a2-0b642d84e780 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:30.802785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7bc51-1327-41f6-92a2-0b642d84e780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:30.833774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1be7bc51-1327-41f6-92a2-0b642d84e780 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:30.835306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1be7bc51-1327-41f6-92a2-0b642d84e780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:30.909744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:30.931359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 180/363] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:27:30.987399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824158-2253-47ca-86cf-90ffe2f0be61 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:30.988028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824158-2253-47ca-86cf-90ffe2f0be61 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:30.988509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824158-2253-47ca-86cf-90ffe2f0be61 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:30.990368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 184/364] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:30.995775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb5398-c658-4fee-a27e-aba97778bb32 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:30.996346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb5398-c658-4fee-a27e-aba97778bb32 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:30.996821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb5398-c658-4fee-a27e-aba97778bb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:31.021798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-20fb5398-c658-4fee-a27e-aba97778bb32 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:31.023340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-20fb5398-c658-4fee-a27e-aba97778bb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:31.103916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 181/365] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:31.287658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abba380-193a-4ae0-85fa-1429b2145c72 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:31.287994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abba380-193a-4ae0-85fa-1429b2145c72 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:31.288179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abba380-193a-4ae0-85fa-1429b2145c72 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:31.289419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb5398-c658-4fee-a27e-aba97778bb32 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:31.290400 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 185/366] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:31.297289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c98e8-9bb8-4118-a998-1010bc57ef14 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:31.297567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c98e8-9bb8-4118-a998-1010bc57ef14 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:31.297842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c98e8-9bb8-4118-a998-1010bc57ef14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:31.322298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-778c98e8-9bb8-4118-a998-1010bc57ef14 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:31.323832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-778c98e8-9bb8-4118-a998-1010bc57ef14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:31.453940 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 186/367] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:27:31.472648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb5398-c658-4fee-a27e-aba97778bb32 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:31.490632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:31.512990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcccf39-bdab-493a-a239-8849c612795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:31.514508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcccf39-bdab-493a-a239-8849c612795d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:31.515017 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcccf39-bdab-493a-a239-8849c612795d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:31.515492 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcccf39-bdab-493a-a239-8849c612795d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:31.535881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcccf39-bdab-493a-a239-8849c612795d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-cdcccf39-bdab-493a-a239-8849c612795d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:31.541249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:31.542111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:31.542885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:31.550756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 182/368] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:31.556769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c98e8-9bb8-4118-a998-1010bc57ef14 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:31.568105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:31.591250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcccf39-bdab-493a-a239-8849c612795d None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:31.751257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c98e8-9bb8-4118-a998-1010bc57ef14 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:31.753919 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcccf39-bdab-493a-a239-8849c612795d None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:31.790488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52553585-5cc9-4ac8-985d-481eeb0ca525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:31.791036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5914d9b-d4b3-4d8f-9488-1124f4aaaa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:31.791169 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52553585-5cc9-4ac8-985d-481eeb0ca525 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:31.791384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52553585-5cc9-4ac8-985d-481eeb0ca525 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:31.791658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52553585-5cc9-4ac8-985d-481eeb0ca525 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:31.792205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5914d9b-d4b3-4d8f-9488-1124f4aaaa4f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:31.792205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5914d9b-d4b3-4d8f-9488-1124f4aaaa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:31.792205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5914d9b-d4b3-4d8f-9488-1124f4aaaa4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:31.792391 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52553585-5cc9-4ac8-985d-481eeb0ca525 None admin] RBAC: Adding query filter params (name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:31.792622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52553585-5cc9-4ac8-985d-481eeb0ca525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:31.794291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52553585-5cc9-4ac8-985d-481eeb0ca525 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:31.808237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 187/369] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:31.808404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5914d9b-d4b3-4d8f-9488-1124f4aaaa4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:31.808803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95266733-ca90-4d38-aac3-ae7a8743cfc9 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:31.808803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95266733-ca90-4d38-aac3-ae7a8743cfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:31.808803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95266733-ca90-4d38-aac3-ae7a8743cfc9 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:31.808803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 188/370] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:31.808803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95266733-ca90-4d38-aac3-ae7a8743cfc9 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-e5914d9b-d4b3-4d8f-9488-1124f4aaaa4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:31.809296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:31.810112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:31.811485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 183/371] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:31.812142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5914d9b-d4b3-4d8f-9488-1124f4aaaa4f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:31.852291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:31.888674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:31.920921 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5914d9b-d4b3-4d8f-9488-1124f4aaaa4f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:31.922379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95266733-ca90-4d38-aac3-ae7a8743cfc9 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:31.959018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0702ff-c367-4594-9ad4-cc6923b7338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:31.959676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0702ff-c367-4594-9ad4-cc6923b7338d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:31.959954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0702ff-c367-4594-9ad4-cc6923b7338d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:31.960223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0702ff-c367-4594-9ad4-cc6923b7338d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:31.966438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76346c0b-d413-4f88-89fa-08583bd03169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:31.967619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76346c0b-d413-4f88-89fa-08583bd03169 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:31.968531 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76346c0b-d413-4f88-89fa-08583bd03169 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:31.968809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76346c0b-d413-4f88-89fa-08583bd03169 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:31.972218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0702ff-c367-4594-9ad4-cc6923b7338d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:31.973653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0702ff-c367-4594-9ad4-cc6923b7338d None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:31.983321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76346c0b-d413-4f88-89fa-08583bd03169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-76346c0b-d413-4f88-89fa-08583bd03169 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:31.989559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:31.990588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:31.991387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-8b0702ff-c367-4594-9ad4-cc6923b7338d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:31.992058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:31.992889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:31.993726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:31.993726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:31.993726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:31.993726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 184/372] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:31.993726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b6712-264f-47eb-90bc-278d82df2b6e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:31.993726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b6712-264f-47eb-90bc-278d82df2b6e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:31.993726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b6712-264f-47eb-90bc-278d82df2b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:31.995692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 189/373] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:31.996511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76346c0b-d413-4f88-89fa-08583bd03169 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.013810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-611b6712-264f-47eb-90bc-278d82df2b6e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:32.015860 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-611b6712-264f-47eb-90bc-278d82df2b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:32.094977 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 185/374] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:32.095846 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b6712-264f-47eb-90bc-278d82df2b6e None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.115930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:32.126552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76346c0b-d413-4f88-89fa-08583bd03169 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:32.162242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:32.168693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a7fca-4bb6-4abb-bf90-f563035406e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:32.169424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a7fca-4bb6-4abb-bf90-f563035406e3 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.169714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a7fca-4bb6-4abb-bf90-f563035406e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.170628 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a7fca-4bb6-4abb-bf90-f563035406e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.171373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a7fca-4bb6-4abb-bf90-f563035406e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:32.171749 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a7fca-4bb6-4abb-bf90-f563035406e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:32.173232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a7fca-4bb6-4abb-bf90-f563035406e3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:32.183399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 190/375] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:32.186422 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a7fca-4bb6-4abb-bf90-f563035406e3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.236223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b6712-264f-47eb-90bc-278d82df2b6e None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:32.276443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcc78e-f44d-4c42-800c-94a0df020ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:32.276989 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcc78e-f44d-4c42-800c-94a0df020ced None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.277275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcc78e-f44d-4c42-800c-94a0df020ced None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.277549 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcc78e-f44d-4c42-800c-94a0df020ced None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.278254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcc78e-f44d-4c42-800c-94a0df020ced None admin] RBAC: Adding query filter params (name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:32.278559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcc78e-f44d-4c42-800c-94a0df020ced None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:32.279985 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcc78e-f44d-4c42-800c-94a0df020ced None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:32.289194 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 186/376] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:32.290457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030191a-bb8d-4c2f-bdcd-e2e6cbdc2275 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.290825 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030191a-bb8d-4c2f-bdcd-e2e6cbdc2275 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.291106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030191a-bb8d-4c2f-bdcd-e2e6cbdc2275 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.297019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 187/377] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:32.298569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030191a-bb8d-4c2f-bdcd-e2e6cbdc2275 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.329991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a7fca-4bb6-4abb-bf90-f563035406e3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:32.367988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ffbdd-f878-4f05-97f5-08b26319fbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:32.368597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ffbdd-f878-4f05-97f5-08b26319fbba None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.368893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ffbdd-f878-4f05-97f5-08b26319fbba None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.369172 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ffbdd-f878-4f05-97f5-08b26319fbba None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.369813 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ffbdd-f878-4f05-97f5-08b26319fbba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:32.370120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ffbdd-f878-4f05-97f5-08b26319fbba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:32.371502 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ffbdd-f878-4f05-97f5-08b26319fbba None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:32.383930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 191/378] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:32.384753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ffbdd-f878-4f05-97f5-08b26319fbba None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.386402 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:32.406966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030191a-bb8d-4c2f-bdcd-e2e6cbdc2275 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:32.438236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:32.441297 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5e259-20b0-429f-83c0-f110eaeecc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:32.441930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5e259-20b0-429f-83c0-f110eaeecc51 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.442210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5e259-20b0-429f-83c0-f110eaeecc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.442474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5e259-20b0-429f-83c0-f110eaeecc51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.454348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5e259-20b0-429f-83c0-f110eaeecc51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:32.455781 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5e259-20b0-429f-83c0-f110eaeecc51 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-29e5e259-20b0-429f-83c0-f110eaeecc51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:32.470256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:32.471173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:32.472103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:32.472103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:32.472103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:32.472103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 188/379] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:32.472770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259ad52-0724-4941-85bc-21d6ce6b157b None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.472770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259ad52-0724-4941-85bc-21d6ce6b157b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.472981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259ad52-0724-4941-85bc-21d6ce6b157b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.488572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ffbdd-f878-4f05-97f5-08b26319fbba None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:32.491351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-9259ad52-0724-4941-85bc-21d6ce6b157b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:32.492527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9259ad52-0724-4941-85bc-21d6ce6b157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:32.522314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6edd9-cf4e-47c2-bde4-1657f688cb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:32.522947 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6edd9-cf4e-47c2-bde4-1657f688cb8f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.523224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6edd9-cf4e-47c2-bde4-1657f688cb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.523498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6edd9-cf4e-47c2-bde4-1657f688cb8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.534192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6edd9-cf4e-47c2-bde4-1657f688cb8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:32.535573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6edd9-cf4e-47c2-bde4-1657f688cb8f None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-d2a6edd9-cf4e-47c2-bde4-1657f688cb8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:32.550561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:32.551491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:32.553134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:32.553134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:32.553134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:32.553134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 192/380] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:32.553134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6edd9-cf4e-47c2-bde4-1657f688cb8f None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.561750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 189/381] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:32.562413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259ad52-0724-4941-85bc-21d6ce6b157b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.634850 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:32.665642 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6edd9-cf4e-47c2-bde4-1657f688cb8f None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:32.669016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:32.677933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259ad52-0724-4941-85bc-21d6ce6b157b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:32.707380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e27eda-61c0-46d6-8966-7cf101deb569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:32.708181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e27eda-61c0-46d6-8966-7cf101deb569 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.708763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e27eda-61c0-46d6-8966-7cf101deb569 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.709183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e27eda-61c0-46d6-8966-7cf101deb569 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.713621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ee5a3-50ce-49df-ba67-95af4b546b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:32.714243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ee5a3-50ce-49df-ba67-95af4b546b14 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.714513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ee5a3-50ce-49df-ba67-95af4b546b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.714818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ee5a3-50ce-49df-ba67-95af4b546b14 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.720193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e27eda-61c0-46d6-8966-7cf101deb569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:32.722407 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ee5a3-50ce-49df-ba67-95af4b546b14 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:32.723848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ee5a3-50ce-49df-ba67-95af4b546b14 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-06e27eda-61c0-46d6-8966-7cf101deb569 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:32.725157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:32.726029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:32.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 193/382] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:32.727482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e27eda-61c0-46d6-8966-7cf101deb569 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-974ee5a3-50ce-49df-ba67-95af4b546b14 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:32.744445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:32.745599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:32.746689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:32.746689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:32.746689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:32.746689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:32.746689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:32.746689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:32.746689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:32.746689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:32.746689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:27:32.746689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:32.748035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 190/383] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Oct 18 11:27:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:32.748877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ee5a3-50ce-49df-ba67-95af4b546b14 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.839968 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e27eda-61c0-46d6-8966-7cf101deb569 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:32.855808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ee5a3-50ce-49df-ba67-95af4b546b14 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:32.874091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08114967-d8c0-4159-b393-68a4618b21e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:32.874704 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08114967-d8c0-4159-b393-68a4618b21e3 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.874980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08114967-d8c0-4159-b393-68a4618b21e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.875251 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08114967-d8c0-4159-b393-68a4618b21e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.875890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08114967-d8c0-4159-b393-68a4618b21e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:32.876191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08114967-d8c0-4159-b393-68a4618b21e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:32.877372 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:32.877595 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08114967-d8c0-4159-b393-68a4618b21e3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:32.888729 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 194/384] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:32.890098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08114967-d8c0-4159-b393-68a4618b21e3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.890251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0c218-5ffd-44f4-8924-76a207d2e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:32.890951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0c218-5ffd-44f4-8924-76a207d2e086 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:32.891321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0c218-5ffd-44f4-8924-76a207d2e086 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:32.891683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0c218-5ffd-44f4-8924-76a207d2e086 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:32.892423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0c218-5ffd-44f4-8924-76a207d2e086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:32.892832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0c218-5ffd-44f4-8924-76a207d2e086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:32.894322 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0c218-5ffd-44f4-8924-76a207d2e086 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:32.906723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 191/385] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:32.911894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0c218-5ffd-44f4-8924-76a207d2e086 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:32.918230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:32.993871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08114967-d8c0-4159-b393-68a4618b21e3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.009739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0c218-5ffd-44f4-8924-76a207d2e086 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.027257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7870-6faf-490e-a971-bb8b524700ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.027875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7870-6faf-490e-a971-bb8b524700ed None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.028156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7870-6faf-490e-a971-bb8b524700ed None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.028429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7870-6faf-490e-a971-bb8b524700ed None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.029128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7870-6faf-490e-a971-bb8b524700ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:33.029461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7870-6faf-490e-a971-bb8b524700ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.030853 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7870-6faf-490e-a971-bb8b524700ed None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.043221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 195/386] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:33.043221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7870-6faf-490e-a971-bb8b524700ed None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.047518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00427d63-2b7c-490a-957d-dadb7dd77781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.048138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00427d63-2b7c-490a-957d-dadb7dd77781 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.048415 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00427d63-2b7c-490a-957d-dadb7dd77781 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.048686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00427d63-2b7c-490a-957d-dadb7dd77781 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.049474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00427d63-2b7c-490a-957d-dadb7dd77781 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:33.049779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00427d63-2b7c-490a-957d-dadb7dd77781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.051217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00427d63-2b7c-490a-957d-dadb7dd77781 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.089668 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 192/387] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Oct 18 11:27:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:33.090589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00427d63-2b7c-490a-957d-dadb7dd77781 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.130071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:33.133684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:33.150631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7870-6faf-490e-a971-bb8b524700ed None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.187802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09aff0-b6d2-42bc-96de-8bd406d35446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.188432 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09aff0-b6d2-42bc-96de-8bd406d35446 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.188701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09aff0-b6d2-42bc-96de-8bd406d35446 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.188998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09aff0-b6d2-42bc-96de-8bd406d35446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.196150 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00427d63-2b7c-490a-957d-dadb7dd77781 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.199541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09aff0-b6d2-42bc-96de-8bd406d35446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.201025 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09aff0-b6d2-42bc-96de-8bd406d35446 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-1e09aff0-b6d2-42bc-96de-8bd406d35446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:33.216077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:33.217394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:33.218764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:33.218764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:33.218764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:33.218764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 196/388] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:33.218764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09aff0-b6d2-42bc-96de-8bd406d35446 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.233487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12be162-d1a4-4808-9316-cbcdc6e3af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.234122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be162-d1a4-4808-9316-cbcdc6e3af56 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.234393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be162-d1a4-4808-9316-cbcdc6e3af56 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.234654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be162-d1a4-4808-9316-cbcdc6e3af56 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.241911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12be162-d1a4-4808-9316-cbcdc6e3af56 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.243352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12be162-d1a4-4808-9316-cbcdc6e3af56 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-c12be162-d1a4-4808-9316-cbcdc6e3af56 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:33.255171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:33.256242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 193/389] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:33.257352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12be162-d1a4-4808-9316-cbcdc6e3af56 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.317825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09aff0-b6d2-42bc-96de-8bd406d35446 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.351095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c94c6-4ad5-4ca8-8851-4e6e7a10c3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.351647 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c94c6-4ad5-4ca8-8851-4e6e7a10c3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.351921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c94c6-4ad5-4ca8-8851-4e6e7a10c3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.352195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c94c6-4ad5-4ca8-8851-4e6e7a10c3f5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.357904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:33.359732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c94c6-4ad5-4ca8-8851-4e6e7a10c3f5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.361176 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12be162-d1a4-4808-9316-cbcdc6e3af56 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.361363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c94c6-4ad5-4ca8-8851-4e6e7a10c3f5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.363173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-584c94c6-4ad5-4ca8-8851-4e6e7a10c3f5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:33.375898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:33.376807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:33.377732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 197/390] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:33.378281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c94c6-4ad5-4ca8-8851-4e6e7a10c3f5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.400962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669da5e7-7cfa-4a57-9ba1-7138a5316807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.401599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da5e7-7cfa-4a57-9ba1-7138a5316807 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.401877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da5e7-7cfa-4a57-9ba1-7138a5316807 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.402151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da5e7-7cfa-4a57-9ba1-7138a5316807 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.412488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669da5e7-7cfa-4a57-9ba1-7138a5316807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.413957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669da5e7-7cfa-4a57-9ba1-7138a5316807 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-669da5e7-7cfa-4a57-9ba1-7138a5316807 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:33.427777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:33.428766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:33.434251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:33.434251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:33.434251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:33.434251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 194/391] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:33.434251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669da5e7-7cfa-4a57-9ba1-7138a5316807 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.479717 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c94c6-4ad5-4ca8-8851-4e6e7a10c3f5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.514666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224eed9d-7af2-4617-9776-1fe3395d59ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.515290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eed9d-7af2-4617-9776-1fe3395d59ea None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.515562 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eed9d-7af2-4617-9776-1fe3395d59ea None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.516254 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eed9d-7af2-4617-9776-1fe3395d59ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.526753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224eed9d-7af2-4617-9776-1fe3395d59ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.527010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669da5e7-7cfa-4a57-9ba1-7138a5316807 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.528229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224eed9d-7af2-4617-9776-1fe3395d59ea None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-224eed9d-7af2-4617-9776-1fe3395d59ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:33.542511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:33.543488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:33.544319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:33.544319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:33.544319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:33.544319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 198/392] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:33.544539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224eed9d-7af2-4617-9776-1fe3395d59ea None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.560018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308c70c-db51-4880-9896-24d2476365f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.560629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308c70c-db51-4880-9896-24d2476365f1 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.561804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308c70c-db51-4880-9896-24d2476365f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.562107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308c70c-db51-4880-9896-24d2476365f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.562746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308c70c-db51-4880-9896-24d2476365f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:33.563048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308c70c-db51-4880-9896-24d2476365f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.564419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308c70c-db51-4880-9896-24d2476365f1 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.580776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 195/393] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:33.580776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308c70c-db51-4880-9896-24d2476365f1 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.591893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:33.604192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:33.658315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224eed9d-7af2-4617-9776-1fe3395d59ea None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.678757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308c70c-db51-4880-9896-24d2476365f1 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.692009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b949ab-3ffe-47b4-b311-df17c8781d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.692615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b949ab-3ffe-47b4-b311-df17c8781d1f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.692916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b949ab-3ffe-47b4-b311-df17c8781d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.693197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b949ab-3ffe-47b4-b311-df17c8781d1f None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.693923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b949ab-3ffe-47b4-b311-df17c8781d1f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:33.694223 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b949ab-3ffe-47b4-b311-df17c8781d1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.695664 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b949ab-3ffe-47b4-b311-df17c8781d1f None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.714083 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08033706-93a8-4343-ac3d-a6ed75988dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.714693 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08033706-93a8-4343-ac3d-a6ed75988dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.714974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08033706-93a8-4343-ac3d-a6ed75988dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.715247 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08033706-93a8-4343-ac3d-a6ed75988dd8 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.715973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08033706-93a8-4343-ac3d-a6ed75988dd8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:33.716278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08033706-93a8-4343-ac3d-a6ed75988dd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.718315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08033706-93a8-4343-ac3d-a6ed75988dd8 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.732805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 199/394] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:33.733681 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b949ab-3ffe-47b4-b311-df17c8781d1f None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.739815 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 196/395] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:33.739815 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08033706-93a8-4343-ac3d-a6ed75988dd8 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.807123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:33.834521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b949ab-3ffe-47b4-b311-df17c8781d1f None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.839145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08033706-93a8-4343-ac3d-a6ed75988dd8 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.844198 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:33.868988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010c70d-c396-4bd7-91e1-ca44af7e31a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.869591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010c70d-c396-4bd7-91e1-ca44af7e31a4 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.869870 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010c70d-c396-4bd7-91e1-ca44af7e31a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.870141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010c70d-c396-4bd7-91e1-ca44af7e31a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.870782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010c70d-c396-4bd7-91e1-ca44af7e31a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:33.871082 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010c70d-c396-4bd7-91e1-ca44af7e31a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.872466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010c70d-c396-4bd7-91e1-ca44af7e31a4 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.873431 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d26dc-c9df-4a8e-a749-c1885073609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:33.874195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d26dc-c9df-4a8e-a749-c1885073609a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:33.874471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d26dc-c9df-4a8e-a749-c1885073609a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:33.875168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d26dc-c9df-4a8e-a749-c1885073609a None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:33.875809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d26dc-c9df-4a8e-a749-c1885073609a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:33.876109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d26dc-c9df-4a8e-a749-c1885073609a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:33.877498 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d26dc-c9df-4a8e-a749-c1885073609a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:33.883764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 200/396] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:33.884342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010c70d-c396-4bd7-91e1-ca44af7e31a4 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.893564 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 197/397] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:33.894369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d26dc-c9df-4a8e-a749-c1885073609a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:33.985286 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010c70d-c396-4bd7-91e1-ca44af7e31a4 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:33.991385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d26dc-c9df-4a8e-a749-c1885073609a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.019449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dca7b-433f-4567-806e-e4a7b96439d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.020073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dca7b-433f-4567-806e-e4a7b96439d3 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.020186 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:34.020360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dca7b-433f-4567-806e-e4a7b96439d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.020628 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dca7b-433f-4567-806e-e4a7b96439d3 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.021393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dca7b-433f-4567-806e-e4a7b96439d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.022938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dca7b-433f-4567-806e-e4a7b96439d3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.026415 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63caf50-58c3-4f2a-9eb1-3e75faad70bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.027053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63caf50-58c3-4f2a-9eb1-3e75faad70bd None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.027331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63caf50-58c3-4f2a-9eb1-3e75faad70bd None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.027599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63caf50-58c3-4f2a-9eb1-3e75faad70bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.039297 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63caf50-58c3-4f2a-9eb1-3e75faad70bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.040795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63caf50-58c3-4f2a-9eb1-3e75faad70bd None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.044497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 201/398] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:27:33 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:27:34.045489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dca7b-433f-4567-806e-e4a7b96439d3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-f63caf50-58c3-4f2a-9eb1-3e75faad70bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:34.056558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:34.057654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:34.058509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:34.058509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:34.058509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:34.058509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 198/399] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:34.058723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63caf50-58c3-4f2a-9eb1-3e75faad70bd None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.090615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:34.188811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63caf50-58c3-4f2a-9eb1-3e75faad70bd None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.192981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dca7b-433f-4567-806e-e4a7b96439d3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.225048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17430290-732b-49c9-9205-be8ad71f6e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.225681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17430290-732b-49c9-9205-be8ad71f6e66 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.225955 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17430290-732b-49c9-9205-be8ad71f6e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.226222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17430290-732b-49c9-9205-be8ad71f6e66 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.231010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04cf08-e90a-471c-8ca8-0dc07ce0bea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.231648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04cf08-e90a-471c-8ca8-0dc07ce0bea7 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.231935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04cf08-e90a-471c-8ca8-0dc07ce0bea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.232205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04cf08-e90a-471c-8ca8-0dc07ce0bea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.234796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17430290-732b-49c9-9205-be8ad71f6e66 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.236226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17430290-732b-49c9-9205-be8ad71f6e66 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.244590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04cf08-e90a-471c-8ca8-0dc07ce0bea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.246106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04cf08-e90a-471c-8ca8-0dc07ce0bea7 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-17430290-732b-49c9-9205-be8ad71f6e66 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:34.247845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:34.248722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 199/400] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:34.250243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17430290-732b-49c9-9205-be8ad71f6e66 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-7f04cf08-e90a-471c-8ca8-0dc07ce0bea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:34.262280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:34.263387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:34.264372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:34.264372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:34.264372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:34.264372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 202/401] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:34.264636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04cf08-e90a-471c-8ca8-0dc07ce0bea7 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.298476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:34.348002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:34.367136 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17430290-732b-49c9-9205-be8ad71f6e66 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.371353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04cf08-e90a-471c-8ca8-0dc07ce0bea7 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.403645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c294e-c529-4cf3-be51-e5097497800d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.404279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c294e-c529-4cf3-be51-e5097497800d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.405162 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c294e-c529-4cf3-be51-e5097497800d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.405424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c294e-c529-4cf3-be51-e5097497800d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.407103 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3d97d-1693-4655-81ee-39ed636c7974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.407714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3d97d-1693-4655-81ee-39ed636c7974 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.407992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3d97d-1693-4655-81ee-39ed636c7974 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.408267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3d97d-1693-4655-81ee-39ed636c7974 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.408943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3d97d-1693-4655-81ee-39ed636c7974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:34.409254 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3d97d-1693-4655-81ee-39ed636c7974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.410634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3d97d-1693-4655-81ee-39ed636c7974 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.413370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c294e-c529-4cf3-be51-e5097497800d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.415003 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c294e-c529-4cf3-be51-e5097497800d None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.422716 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 203/402] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:34.423590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3d97d-1693-4655-81ee-39ed636c7974 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-d76c294e-c529-4cf3-be51-e5097497800d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:34.429041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:34.430108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:34.431159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:34.431159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:34.431159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:34.431159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:34.431159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:34.431159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:34.431159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:34.431159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:34.431159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 200/403] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:34.431159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c294e-c529-4cf3-be51-e5097497800d None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.538782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3d97d-1693-4655-81ee-39ed636c7974 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.553088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c294e-c529-4cf3-be51-e5097497800d None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.554783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:34.573253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d1e19-3883-43e8-a9f0-008c3fefefe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.573867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d1e19-3883-43e8-a9f0-008c3fefefe6 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.574146 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d1e19-3883-43e8-a9f0-008c3fefefe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.574417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d1e19-3883-43e8-a9f0-008c3fefefe6 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.575149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d1e19-3883-43e8-a9f0-008c3fefefe6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:34.575452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d1e19-3883-43e8-a9f0-008c3fefefe6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.577645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d1e19-3883-43e8-a9f0-008c3fefefe6 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.581126 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:34.594490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864ea64-a6d9-4168-99eb-6aae199c4fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.595108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864ea64-a6d9-4168-99eb-6aae199c4fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.595389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864ea64-a6d9-4168-99eb-6aae199c4fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.595661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864ea64-a6d9-4168-99eb-6aae199c4fa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.596298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864ea64-a6d9-4168-99eb-6aae199c4fa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:34.596598 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864ea64-a6d9-4168-99eb-6aae199c4fa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.597951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 204/404] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:34.598585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d1e19-3883-43e8-a9f0-008c3fefefe6 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.602601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864ea64-a6d9-4168-99eb-6aae199c4fa6 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.622198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 201/405] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:34.622198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864ea64-a6d9-4168-99eb-6aae199c4fa6 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.713995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d1e19-3883-43e8-a9f0-008c3fefefe6 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.728399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864ea64-a6d9-4168-99eb-6aae199c4fa6 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.759837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd3ff5-0041-4bfa-8417-fea9d78145d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.760473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd3ff5-0041-4bfa-8417-fea9d78145d5 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.760749 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd3ff5-0041-4bfa-8417-fea9d78145d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.761070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd3ff5-0041-4bfa-8417-fea9d78145d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.765629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6801a-b477-47a1-bba3-2cc66b4760f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.766237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6801a-b477-47a1-bba3-2cc66b4760f8 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.766516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6801a-b477-47a1-bba3-2cc66b4760f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.766786 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6801a-b477-47a1-bba3-2cc66b4760f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.767647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6801a-b477-47a1-bba3-2cc66b4760f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:34.767914 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd3ff5-0041-4bfa-8417-fea9d78145d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.768008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6801a-b477-47a1-bba3-2cc66b4760f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.769392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd3ff5-0041-4bfa-8417-fea9d78145d5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.769506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6801a-b477-47a1-bba3-2cc66b4760f8 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-cccd3ff5-0041-4bfa-8417-fea9d78145d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:34.782290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:34.783178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:34.789734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:34.789734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:34.789734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:34.789734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:34.789734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:34.789734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:34.789734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:34.789734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:34.789734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 205/406] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:34.789734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd3ff5-0041-4bfa-8417-fea9d78145d5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.792670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 202/407] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:34.793756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6801a-b477-47a1-bba3-2cc66b4760f8 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.822102 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:34.845981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:34.893009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd3ff5-0041-4bfa-8417-fea9d78145d5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.908751 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6801a-b477-47a1-bba3-2cc66b4760f8 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:34.927484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1dacc-8ffc-4394-bd90-2ec93fe11996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.928121 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1dacc-8ffc-4394-bd90-2ec93fe11996 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.928398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1dacc-8ffc-4394-bd90-2ec93fe11996 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.928671 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1dacc-8ffc-4394-bd90-2ec93fe11996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.939449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1dacc-8ffc-4394-bd90-2ec93fe11996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.940953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1dacc-8ffc-4394-bd90-2ec93fe11996 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.946174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91287c9c-d66b-4540-8a79-1c3f30ca8008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:34.947121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91287c9c-d66b-4540-8a79-1c3f30ca8008 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:34.947548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91287c9c-d66b-4540-8a79-1c3f30ca8008 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:34.948102 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91287c9c-d66b-4540-8a79-1c3f30ca8008 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:34.955928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91287c9c-d66b-4540-8a79-1c3f30ca8008 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-a1e1dacc-8ffc-4394-bd90-2ec93fe11996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:34.956748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:34.957834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:34.958679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:34.958679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:34.958679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:34.958838 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91287c9c-d66b-4540-8a79-1c3f30ca8008 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:34.958909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 206/408] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:34.959484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1dacc-8ffc-4394-bd90-2ec93fe11996 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-91287c9c-d66b-4540-8a79-1c3f30ca8008 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:34.975933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:34.976870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:34.977727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:34.977727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:34.977727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:34.977727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:34.977727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:34.977727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:34.977727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:34.977727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:34.977727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 203/409] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:34.978702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91287c9c-d66b-4540-8a79-1c3f30ca8008 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:35.068887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:35.078333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1dacc-8ffc-4394-bd90-2ec93fe11996 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:35.087622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:35.088080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91287c9c-d66b-4540-8a79-1c3f30ca8008 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:35.117756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e985d5-53f7-477e-a488-f55403647202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:35.118520 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e985d5-53f7-477e-a488-f55403647202 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:35.118896 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e985d5-53f7-477e-a488-f55403647202 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:35.119274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e985d5-53f7-477e-a488-f55403647202 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:35.120083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e985d5-53f7-477e-a488-f55403647202 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:35.120486 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e985d5-53f7-477e-a488-f55403647202 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:35.122044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e985d5-53f7-477e-a488-f55403647202 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:35.125068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92304f5f-45c4-4be8-b4f0-b0cc04503f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:35.125727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92304f5f-45c4-4be8-b4f0-b0cc04503f99 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:35.126011 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92304f5f-45c4-4be8-b4f0-b0cc04503f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:35.126292 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92304f5f-45c4-4be8-b4f0-b0cc04503f99 None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/87f8a4a8277745a9ad1976af9df31cb9/roles/f94c9d39cd6244b4b0a727e200ff2e94` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:35.140610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 207/410] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:35.143633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92304f5f-45c4-4be8-b4f0-b0cc04503f99 None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=87f8a4a8277745a9ad1976af9df31cb9, role_id=f94c9d39cd6244b4b0a727e200ff2e94)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:35.145116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92304f5f-45c4-4be8-b4f0-b0cc04503f99 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:35.146083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e985d5-53f7-477e-a488-f55403647202 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:35.159729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 204/411] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:27:34 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/87f8a4a8277745a9ad1976af9df31cb9/roles/f94c9d39cd6244b4b0a727e200ff2e94 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:35.160482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92304f5f-45c4-4be8-b4f0-b0cc04503f99 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:35.284577 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e985d5-53f7-477e-a488-f55403647202 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:35.299074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92304f5f-45c4-4be8-b4f0-b0cc04503f99 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:35.328536 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232cb19-0dce-4231-a067-004f8fe95d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:35.329173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232cb19-0dce-4231-a067-004f8fe95d1e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:35.329456 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232cb19-0dce-4231-a067-004f8fe95d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:35.329732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232cb19-0dce-4231-a067-004f8fe95d1e None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:35.330431 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232cb19-0dce-4231-a067-004f8fe95d1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:35.331086 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232cb19-0dce-4231-a067-004f8fe95d1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:35.332878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232cb19-0dce-4231-a067-004f8fe95d1e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:35.343252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:35.351284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 208/412] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:35.352151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232cb19-0dce-4231-a067-004f8fe95d1e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:35.352502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccac5f3-a1c9-4a61-ac88-fd54ffc33b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:35.353341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccac5f3-a1c9-4a61-ac88-fd54ffc33b00 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:35.353617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccac5f3-a1c9-4a61-ac88-fd54ffc33b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:35.353883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccac5f3-a1c9-4a61-ac88-fd54ffc33b00 None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:35.354730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccac5f3-a1c9-4a61-ac88-fd54ffc33b00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:35.355034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccac5f3-a1c9-4a61-ac88-fd54ffc33b00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:35.357097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccac5f3-a1c9-4a61-ac88-fd54ffc33b00 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:35.375259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 205/413] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:35.376881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccac5f3-a1c9-4a61-ac88-fd54ffc33b00 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:35.398347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:35.509852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccac5f3-a1c9-4a61-ac88-fd54ffc33b00 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:35.510755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232cb19-0dce-4231-a067-004f8fe95d1e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:35.558593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4a1b7-fdde-4e3a-9691-1a23597ca631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:35.559495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4a1b7-fdde-4e3a-9691-1a23597ca631 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:35.560085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4a1b7-fdde-4e3a-9691-1a23597ca631 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:35.561117 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4a1b7-fdde-4e3a-9691-1a23597ca631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:35.561886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b21df-021a-4e00-a244-48c6f03b065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:35.562671 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b21df-021a-4e00-a244-48c6f03b065a None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:35.563069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b21df-021a-4e00-a244-48c6f03b065a None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:35.563412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b21df-021a-4e00-a244-48c6f03b065a None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/3d2a655f4d6f45039d641133e3b56592/roles/f94c9d39cd6244b4b0a727e200ff2e94` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:35.571005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4a1b7-fdde-4e3a-9691-1a23597ca631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:35.572754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4a1b7-fdde-4e3a-9691-1a23597ca631 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:35.583321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b21df-021a-4e00-a244-48c6f03b065a None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=3d2a655f4d6f45039d641133e3b56592, role_id=f94c9d39cd6244b4b0a727e200ff2e94)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:35.584802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b21df-021a-4e00-a244-48c6f03b065a None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-07c4a1b7-fdde-4e3a-9691-1a23597ca631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:35.595266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:35.597001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:35.598385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:35.598385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:35.598385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:35.598385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:35.598385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:35.598385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:35.598385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:35.598385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:35.600435 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 206/414] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:35.601571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4a1b7-fdde-4e3a-9691-1a23597ca631 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:35.601765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 209/415] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:27:35 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/3d2a655f4d6f45039d641133e3b56592/roles/f94c9d39cd6244b4b0a727e200ff2e94 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:35.612873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b21df-021a-4e00-a244-48c6f03b065a None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:35.751538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:35.808223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4a1b7-fdde-4e3a-9691-1a23597ca631 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:35.818454 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:35.852958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3b0a4-a4f9-4eb5-a7cf-c07890a09a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:35.853907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3b0a4-a4f9-4eb5-a7cf-c07890a09a77 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:35.854377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3b0a4-a4f9-4eb5-a7cf-c07890a09a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:35.854481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b21df-021a-4e00-a244-48c6f03b065a None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:35.855244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3b0a4-a4f9-4eb5-a7cf-c07890a09a77 None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/bb051b00a16949798158bac1ac57d36f/roles/f94c9d39cd6244b4b0a727e200ff2e94` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:35.874205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3b0a4-a4f9-4eb5-a7cf-c07890a09a77 None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=bb051b00a16949798158bac1ac57d36f, role_id=f94c9d39cd6244b4b0a727e200ff2e94)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:35.875971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3b0a4-a4f9-4eb5-a7cf-c07890a09a77 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:35.893867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 207/416] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:27:35 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/bb051b00a16949798158bac1ac57d36f/roles/f94c9d39cd6244b4b0a727e200ff2e94 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:35.902989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9394088-8ba9-4260-bec1-6338397c56a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:35.903674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9394088-8ba9-4260-bec1-6338397c56a9 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:35.903957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9394088-8ba9-4260-bec1-6338397c56a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:35.904217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9394088-8ba9-4260-bec1-6338397c56a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:35.905638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9394088-8ba9-4260-bec1-6338397c56a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:35.905953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9394088-8ba9-4260-bec1-6338397c56a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:35.907570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9394088-8ba9-4260-bec1-6338397c56a9 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:35.932620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 210/417] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:35.938719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3b0a4-a4f9-4eb5-a7cf-c07890a09a77 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:36.087752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3b0a4-a4f9-4eb5-a7cf-c07890a09a77 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:36.089398 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:36.127744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d78aa1-0280-42ef-8032-782b1313fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:36.128765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d78aa1-0280-42ef-8032-782b1313fe01 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:36.129351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d78aa1-0280-42ef-8032-782b1313fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:36.129825 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d78aa1-0280-42ef-8032-782b1313fe01 None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/d301f3d9aeb644a38c31dbdbef2a48c7/roles/f94c9d39cd6244b4b0a727e200ff2e94` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:36.145651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d78aa1-0280-42ef-8032-782b1313fe01 None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=d301f3d9aeb644a38c31dbdbef2a48c7, role_id=f94c9d39cd6244b4b0a727e200ff2e94)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:36.147601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d78aa1-0280-42ef-8032-782b1313fe01 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:36.165573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 208/418] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:27:35 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/d301f3d9aeb644a38c31dbdbef2a48c7/roles/f94c9d39cd6244b4b0a727e200ff2e94 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:36.295946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec0542-0f45-405c-8d40-609ed15f0af1 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:36.296231 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec0542-0f45-405c-8d40-609ed15f0af1 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:36.296507 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec0542-0f45-405c-8d40-609ed15f0af1 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:36.297953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 211/419] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:36.303800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdcb88-8e4a-4e6c-b598-b52e8be8c7ce None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:36.304091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdcb88-8e4a-4e6c-b598-b52e8be8c7ce None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:36.304344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdcb88-8e4a-4e6c-b598-b52e8be8c7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:36.333514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1bfdcb88-8e4a-4e6c-b598-b52e8be8c7ce None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:36.334780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1bfdcb88-8e4a-4e6c-b598-b52e8be8c7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:36.413013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 209/420] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:27:36.696274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec0542-0f45-405c-8d40-609ed15f0af1 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:36.838852 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec0542-0f45-405c-8d40-609ed15f0af1 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:36.878770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609787b8-059b-4065-a1a3-c6defe7bd321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:36.879420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609787b8-059b-4065-a1a3-c6defe7bd321 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:36.879689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609787b8-059b-4065-a1a3-c6defe7bd321 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:36.879962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609787b8-059b-4065-a1a3-c6defe7bd321 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:36.883314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:36.893178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609787b8-059b-4065-a1a3-c6defe7bd321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-609787b8-059b-4065-a1a3-c6defe7bd321 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:36.898073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:36.898955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:36.899763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 212/421] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:36.905923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfdcb88-8e4a-4e6c-b598-b52e8be8c7ce None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:37.058231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:37.083076 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfdcb88-8e4a-4e6c-b598-b52e8be8c7ce None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:37.131594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c36834-3da3-40a2-a24f-9f99510d67a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:37.132211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c36834-3da3-40a2-a24f-9f99510d67a3 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:37.132485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c36834-3da3-40a2-a24f-9f99510d67a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:37.132752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c36834-3da3-40a2-a24f-9f99510d67a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:37.133478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c36834-3da3-40a2-a24f-9f99510d67a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:37.133775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c36834-3da3-40a2-a24f-9f99510d67a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:37.135138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c36834-3da3-40a2-a24f-9f99510d67a3 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:37.145088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 210/422] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:37.150708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609787b8-059b-4065-a1a3-c6defe7bd321 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:37.287825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609787b8-059b-4065-a1a3-c6defe7bd321 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:37.331248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720bf77-8bdb-4254-8e93-371da9b26b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:37.331801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720bf77-8bdb-4254-8e93-371da9b26b5e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:37.332076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720bf77-8bdb-4254-8e93-371da9b26b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:37.332395 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720bf77-8bdb-4254-8e93-371da9b26b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:37.343004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720bf77-8bdb-4254-8e93-371da9b26b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:37.344542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720bf77-8bdb-4254-8e93-371da9b26b5e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-a720bf77-8bdb-4254-8e93-371da9b26b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:37.361280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:37.362202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:37.363029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:37.363029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:37.363029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:37.363029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 213/423] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:37.368701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c36834-3da3-40a2-a24f-9f99510d67a3 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:37.515802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c36834-3da3-40a2-a24f-9f99510d67a3 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:37.563823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1210a0-4a9c-4ccb-b62c-729d119718dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:37.564482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1210a0-4a9c-4ccb-b62c-729d119718dc None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:37.564760 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1210a0-4a9c-4ccb-b62c-729d119718dc None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:37.565108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1210a0-4a9c-4ccb-b62c-729d119718dc None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:37.565757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1210a0-4a9c-4ccb-b62c-729d119718dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:37.566053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1210a0-4a9c-4ccb-b62c-729d119718dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:37.567431 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1210a0-4a9c-4ccb-b62c-729d119718dc None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:37.584553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 211/424] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:37.590097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720bf77-8bdb-4254-8e93-371da9b26b5e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:37.616756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde2de1-a812-42fb-becf-c17ac60559a1 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:37.617089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde2de1-a812-42fb-becf-c17ac60559a1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:37.617314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde2de1-a812-42fb-becf-c17ac60559a1 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:37.618863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 212/425] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:27:37.625442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e31bf-41a6-41c3-9f60-07dafc82e8bf None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:37.626034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e31bf-41a6-41c3-9f60-07dafc82e8bf None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:37.626497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e31bf-41a6-41c3-9f60-07dafc82e8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:37.645220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:37.648177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1d6e31bf-41a6-41c3-9f60-07dafc82e8bf None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:37.649948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1d6e31bf-41a6-41c3-9f60-07dafc82e8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:37.688355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:37.743130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 213/426] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:37.787166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720bf77-8bdb-4254-8e93-371da9b26b5e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:37.823935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab98df2-fa20-4141-bb08-6dcd18fc28cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:37.824575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab98df2-fa20-4141-bb08-6dcd18fc28cc None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:37.824889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab98df2-fa20-4141-bb08-6dcd18fc28cc None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:37.825292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab98df2-fa20-4141-bb08-6dcd18fc28cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:37.833547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab98df2-fa20-4141-bb08-6dcd18fc28cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:37.835075 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab98df2-fa20-4141-bb08-6dcd18fc28cc None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-6ab98df2-fa20-4141-bb08-6dcd18fc28cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:37.848046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:37.849029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:37.850152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 214/427] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:27:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:37.854985 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e31bf-41a6-41c3-9f60-07dafc82e8bf None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:38.004692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e31bf-41a6-41c3-9f60-07dafc82e8bf None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:38.054880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d28ed0-657d-4084-a418-9f90357f7f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:38.055724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d28ed0-657d-4084-a418-9f90357f7f14 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:38.056143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d28ed0-657d-4084-a418-9f90357f7f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:38.056551 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d28ed0-657d-4084-a418-9f90357f7f14 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:38.057455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d28ed0-657d-4084-a418-9f90357f7f14 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:38.057916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d28ed0-657d-4084-a418-9f90357f7f14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:38.059477 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d28ed0-657d-4084-a418-9f90357f7f14 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:38.061795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:38.084507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 214/428] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Oct 18 11:27:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:38.090452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab98df2-fa20-4141-bb08-6dcd18fc28cc None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:38.144080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d28ed0-657d-4084-a418-9f90357f7f14 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:38.260065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d28ed0-657d-4084-a418-9f90357f7f14 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:38.260984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab98df2-fa20-4141-bb08-6dcd18fc28cc None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:38.271204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:38.299107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44433f84-be5d-4529-a60d-2ba1bf8a6aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:38.299749 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433f84-be5d-4529-a60d-2ba1bf8a6aae None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:38.300022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433f84-be5d-4529-a60d-2ba1bf8a6aae None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:38.300350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433f84-be5d-4529-a60d-2ba1bf8a6aae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:38.305067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df11822-8ae0-4d90-b575-14121bd8143c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:38.305990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df11822-8ae0-4d90-b575-14121bd8143c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:38.306472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df11822-8ae0-4d90-b575-14121bd8143c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:38.306956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df11822-8ae0-4d90-b575-14121bd8143c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:38.311887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44433f84-be5d-4529-a60d-2ba1bf8a6aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:38.313411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44433f84-be5d-4529-a60d-2ba1bf8a6aae None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:38.319926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df11822-8ae0-4d90-b575-14121bd8143c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-0df11822-8ae0-4d90-b575-14121bd8143c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:38.327134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:38.328028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:38.328817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-44433f84-be5d-4529-a60d-2ba1bf8a6aae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:38.329441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:38.330226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:38.332111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:38.332111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:38.332111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:38.332111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 215/429] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:38.332111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b501b2b-9967-445b-b8ef-5da183ae5c83 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:38.332111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b501b2b-9967-445b-b8ef-5da183ae5c83 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:38.332111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b501b2b-9967-445b-b8ef-5da183ae5c83 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:38.334567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 215/430] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:38.339340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 216/431] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:38.342846 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b501b2b-9967-445b-b8ef-5da183ae5c83 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:38.343601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df11822-8ae0-4d90-b575-14121bd8143c None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:38.407819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:38.459957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df11822-8ae0-4d90-b575-14121bd8143c None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:38.494656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b501b2b-9967-445b-b8ef-5da183ae5c83 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:38.512098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1910c-df42-4af8-aa77-89a6d20d7247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:38.513464 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1910c-df42-4af8-aa77-89a6d20d7247 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:38.514519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1910c-df42-4af8-aa77-89a6d20d7247 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:38.515441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1910c-df42-4af8-aa77-89a6d20d7247 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:38.516368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1910c-df42-4af8-aa77-89a6d20d7247 None admin] RBAC: Adding query filter params (name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:38.516905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1910c-df42-4af8-aa77-89a6d20d7247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:38.519183 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1910c-df42-4af8-aa77-89a6d20d7247 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:38.536722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 216/432] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:38.536722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1fead-b851-4ab8-9cb4-df2e48547e77 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:38.536722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1fead-b851-4ab8-9cb4-df2e48547e77 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:38.536722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1fead-b851-4ab8-9cb4-df2e48547e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:38.537699 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681c4e5-b7d6-46bb-9188-a535e169b032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:38.538035 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681c4e5-b7d6-46bb-9188-a535e169b032 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:38.538331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681c4e5-b7d6-46bb-9188-a535e169b032 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:38.538610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681c4e5-b7d6-46bb-9188-a535e169b032 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:38.539258 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681c4e5-b7d6-46bb-9188-a535e169b032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:38.539554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681c4e5-b7d6-46bb-9188-a535e169b032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:38.540963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681c4e5-b7d6-46bb-9188-a535e169b032 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:38.558662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 217/433] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:38.559586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681c4e5-b7d6-46bb-9188-a535e169b032 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:38.562563 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-caa1fead-b851-4ab8-9cb4-df2e48547e77 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:38.565326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-caa1fead-b851-4ab8-9cb4-df2e48547e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:38.586074 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:38.665700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 217/434] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:38.666892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1fead-b851-4ab8-9cb4-df2e48547e77 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:38.695064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681c4e5-b7d6-46bb-9188-a535e169b032 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:38.723350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:38.783390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc3b66-1a9b-4457-a34d-b664c0c62ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:38.783761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc3b66-1a9b-4457-a34d-b664c0c62ebc None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:38.784029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc3b66-1a9b-4457-a34d-b664c0c62ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:38.784293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc3b66-1a9b-4457-a34d-b664c0c62ebc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:38.796947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1fead-b851-4ab8-9cb4-df2e48547e77 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:38.811686 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc3b66-1a9b-4457-a34d-b664c0c62ebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:38.813280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc3b66-1a9b-4457-a34d-b664c0c62ebc None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:38.840405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab7f1b-a2c8-4566-9e54-09f083731d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:38.841455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab7f1b-a2c8-4566-9e54-09f083731d7b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:38.842022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab7f1b-a2c8-4566-9e54-09f083731d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:38.842493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab7f1b-a2c8-4566-9e54-09f083731d7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-21dc3b66-1a9b-4457-a34d-b664c0c62ebc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:38.843682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:38.844568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:38.845462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:38.845462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:38.845462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:38.845462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 218/435] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:38.858417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc3b66-1a9b-4457-a34d-b664c0c62ebc None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:38.858812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab7f1b-a2c8-4566-9e54-09f083731d7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:38.860507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab7f1b-a2c8-4566-9e54-09f083731d7b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-b8ab7f1b-a2c8-4566-9e54-09f083731d7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:38.876515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:38.877444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:38.878300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:38.878300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:38.878300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:38.878300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:38.878300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:38.878300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:38.878300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:38.878300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:38.880748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 218/436] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:38.888971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab7f1b-a2c8-4566-9e54-09f083731d7b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:38.958029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:38.995838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc3b66-1a9b-4457-a34d-b664c0c62ebc None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:39.020052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab7f1b-a2c8-4566-9e54-09f083731d7b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:39.024245 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:39.043249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908d893-c61d-4923-b962-dea3657a90d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:39.044162 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908d893-c61d-4923-b962-dea3657a90d8 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.044442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908d893-c61d-4923-b962-dea3657a90d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.044752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908d893-c61d-4923-b962-dea3657a90d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.046347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908d893-c61d-4923-b962-dea3657a90d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:39.046648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908d893-c61d-4923-b962-dea3657a90d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:39.048023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908d893-c61d-4923-b962-dea3657a90d8 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:39.063424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 219/437] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:39.064831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656d5f1-a882-4a30-96c8-3d1208b921c7 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.065146 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656d5f1-a882-4a30-96c8-3d1208b921c7 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.065474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656d5f1-a882-4a30-96c8-3d1208b921c7 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.066864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5d564-d85c-45b9-ba88-0f786f0893f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:39.067997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d564-d85c-45b9-ba88-0f786f0893f7 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.068521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d564-d85c-45b9-ba88-0f786f0893f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.069039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d564-d85c-45b9-ba88-0f786f0893f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.069813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5d564-d85c-45b9-ba88-0f786f0893f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:39.070116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5d564-d85c-45b9-ba88-0f786f0893f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:39.070724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 220/438] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:39.071436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656d5f1-a882-4a30-96c8-3d1208b921c7 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:39.071532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5d564-d85c-45b9-ba88-0f786f0893f7 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:39.085995 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 219/439] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:39.091768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5d564-d85c-45b9-ba88-0f786f0893f7 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:39.187937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656d5f1-a882-4a30-96c8-3d1208b921c7 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:39.207968 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5d564-d85c-45b9-ba88-0f786f0893f7 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:39.220314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:39.226094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df819764-c45a-4222-bd25-86d1423d0219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:39.226972 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df819764-c45a-4222-bd25-86d1423d0219 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.227238 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df819764-c45a-4222-bd25-86d1423d0219 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.227510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df819764-c45a-4222-bd25-86d1423d0219 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.239095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df819764-c45a-4222-bd25-86d1423d0219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-df819764-c45a-4222-bd25-86d1423d0219 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:39.243902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:39.244990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:39.245980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 221/440] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:39.246736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccf20c-aa3f-4cb4-93a2-129e32f67f69 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.246736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccf20c-aa3f-4cb4-93a2-129e32f67f69 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.247010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccf20c-aa3f-4cb4-93a2-129e32f67f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.254550 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54842a5b-fba3-4180-acb6-f6ebcc92839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:39.255336 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54842a5b-fba3-4180-acb6-f6ebcc92839a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.255619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54842a5b-fba3-4180-acb6-f6ebcc92839a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.255891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54842a5b-fba3-4180-acb6-f6ebcc92839a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.264261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54842a5b-fba3-4180-acb6-f6ebcc92839a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:39.265728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54842a5b-fba3-4180-acb6-f6ebcc92839a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:39.267041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-dbccf20c-aa3f-4cb4-93a2-129e32f67f69 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:39.268275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-dbccf20c-aa3f-4cb4-93a2-129e32f67f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-54842a5b-fba3-4180-acb6-f6ebcc92839a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:39.277541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:39.278452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:39.279885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:39.279885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:39.279885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:39.279885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:39.279885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:39.279885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:39.279885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:39.279885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:39.279885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:27:39.279885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:39.286970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 220/441] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Oct 18 11:27:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:39.287829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54842a5b-fba3-4180-acb6-f6ebcc92839a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:39.316152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:39.343596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 222/442] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:39.345044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab51b8-bfdc-4c19-8c17-54fc5b54e6bd None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.345331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab51b8-bfdc-4c19-8c17-54fc5b54e6bd None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.345604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab51b8-bfdc-4c19-8c17-54fc5b54e6bd None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.346850 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 223/443] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:39.347547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab51b8-bfdc-4c19-8c17-54fc5b54e6bd None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:39.417114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54842a5b-fba3-4180-acb6-f6ebcc92839a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:39.454554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3cee4-0e8e-4b46-a040-2db4f9f1f742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:39.455212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3cee4-0e8e-4b46-a040-2db4f9f1f742 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.455493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3cee4-0e8e-4b46-a040-2db4f9f1f742 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.455766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3cee4-0e8e-4b46-a040-2db4f9f1f742 None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/4be9de8292214872a59c1466f7513539/roles/f94c9d39cd6244b4b0a727e200ff2e94` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.467423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab51b8-bfdc-4c19-8c17-54fc5b54e6bd None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:39.473481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3cee4-0e8e-4b46-a040-2db4f9f1f742 None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=4be9de8292214872a59c1466f7513539, role_id=f94c9d39cd6244b4b0a727e200ff2e94)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:39.475131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3cee4-0e8e-4b46-a040-2db4f9f1f742 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:39.492776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:39.500986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 221/444] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:27:39 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/4be9de8292214872a59c1466f7513539/roles/f94c9d39cd6244b4b0a727e200ff2e94 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:39.500986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3cee4-0e8e-4b46-a040-2db4f9f1f742 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:39.514124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9983d09-ef21-4601-ac17-7413d0134d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:39.515138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9983d09-ef21-4601-ac17-7413d0134d3d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.515582 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9983d09-ef21-4601-ac17-7413d0134d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.516043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9983d09-ef21-4601-ac17-7413d0134d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.517032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9983d09-ef21-4601-ac17-7413d0134d3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:39.517478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9983d09-ef21-4601-ac17-7413d0134d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:39.520075 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9983d09-ef21-4601-ac17-7413d0134d3d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:39.537332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 224/445] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:39.539411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7b52e-a8ca-4def-b2da-a095ba89bc89 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.539857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7b52e-a8ca-4def-b2da-a095ba89bc89 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.540306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7b52e-a8ca-4def-b2da-a095ba89bc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.563474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ece7b52e-a8ca-4def-b2da-a095ba89bc89 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:39.565885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ece7b52e-a8ca-4def-b2da-a095ba89bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:39.660610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 225/446] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:39.661903 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7b52e-a8ca-4def-b2da-a095ba89bc89 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:39.670970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:39.700523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3cee4-0e8e-4b46-a040-2db4f9f1f742 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:39.736597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cec082-fa49-4585-9077-be54236dd303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:39.737222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cec082-fa49-4585-9077-be54236dd303 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.737417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cec082-fa49-4585-9077-be54236dd303 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.737681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cec082-fa49-4585-9077-be54236dd303 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.738494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cec082-fa49-4585-9077-be54236dd303 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:39.738724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cec082-fa49-4585-9077-be54236dd303 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:39.740418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cec082-fa49-4585-9077-be54236dd303 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:39.762752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 222/447] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Oct 18 11:27:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:39.763413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cec082-fa49-4585-9077-be54236dd303 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:39.792123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7b52e-a8ca-4def-b2da-a095ba89bc89 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:39.829505 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270b089-b881-4f36-9c50-ccf647f0d71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:39.830150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270b089-b881-4f36-9c50-ccf647f0d71d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.830478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270b089-b881-4f36-9c50-ccf647f0d71d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.830774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270b089-b881-4f36-9c50-ccf647f0d71d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.841353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270b089-b881-4f36-9c50-ccf647f0d71d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:39.843414 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270b089-b881-4f36-9c50-ccf647f0d71d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-6270b089-b881-4f36-9c50-ccf647f0d71d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:39.859340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:39.860301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:39.861263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:39.861263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:39.861263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:39.861263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 226/448] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:39.861263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270b089-b881-4f36-9c50-ccf647f0d71d None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:39.861959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:39.905918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cec082-fa49-4585-9077-be54236dd303 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:39.947637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:39.954399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d5b32-cde3-4e5d-af2a-0f51adbfc46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:39.955177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d5b32-cde3-4e5d-af2a-0f51adbfc46e None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:39.955705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d5b32-cde3-4e5d-af2a-0f51adbfc46e None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:39.956064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d5b32-cde3-4e5d-af2a-0f51adbfc46e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:39.967202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d5b32-cde3-4e5d-af2a-0f51adbfc46e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:39.968449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270b089-b881-4f36-9c50-ccf647f0d71d None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-0d6d5b32-cde3-4e5d-af2a-0f51adbfc46e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:39.972243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:39.973345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:39.974308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:39.975965 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 223/449] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:39.976764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d5b32-cde3-4e5d-af2a-0f51adbfc46e None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.009799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c634fc-902d-4f7c-ade0-9055a71c05e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:40.010438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c634fc-902d-4f7c-ade0-9055a71c05e6 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:40.010718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c634fc-902d-4f7c-ade0-9055a71c05e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:40.010994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c634fc-902d-4f7c-ade0-9055a71c05e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:40.047770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:40.085559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c634fc-902d-4f7c-ade0-9055a71c05e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:40.087116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c634fc-902d-4f7c-ade0-9055a71c05e6 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:40.088718 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d5b32-cde3-4e5d-af2a-0f51adbfc46e None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:40.129763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951381b8-1361-432f-85bd-798b5e181f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:40.130492 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951381b8-1361-432f-85bd-798b5e181f18 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:40.131091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951381b8-1361-432f-85bd-798b5e181f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:40.131366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951381b8-1361-432f-85bd-798b5e181f18 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:40.132295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951381b8-1361-432f-85bd-798b5e181f18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:40.132597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951381b8-1361-432f-85bd-798b5e181f18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:40.133994 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951381b8-1361-432f-85bd-798b5e181f18 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:40.147142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 224/450] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:40.148323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951381b8-1361-432f-85bd-798b5e181f18 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.207193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 227/451] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:27:40.208131 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c634fc-902d-4f7c-ade0-9055a71c05e6 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.212554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:40.264607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951381b8-1361-432f-85bd-798b5e181f18 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:40.276440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:40.302541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cccf84-8bd1-420e-b0a8-148fe16bfa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:40.303180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cccf84-8bd1-420e-b0a8-148fe16bfa17 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:40.303463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cccf84-8bd1-420e-b0a8-148fe16bfa17 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:40.303724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cccf84-8bd1-420e-b0a8-148fe16bfa17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:40.314589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c634fc-902d-4f7c-ade0-9055a71c05e6 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:40.315972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cccf84-8bd1-420e-b0a8-148fe16bfa17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-44cccf84-8bd1-420e-b0a8-148fe16bfa17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:40.321525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:40.322447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:40.323225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 225/452] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:40.323870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cccf84-8bd1-420e-b0a8-148fe16bfa17 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.355210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac7047-430d-4e24-a149-8d5a4c900b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:40.355822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac7047-430d-4e24-a149-8d5a4c900b18 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:40.356097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac7047-430d-4e24-a149-8d5a4c900b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:40.356365 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac7047-430d-4e24-a149-8d5a4c900b18 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:40.357062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac7047-430d-4e24-a149-8d5a4c900b18 None admin] RBAC: Adding query filter params (name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:40.357912 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac7047-430d-4e24-a149-8d5a4c900b18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:40.359271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac7047-430d-4e24-a149-8d5a4c900b18 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:40.368070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 228/453] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:40.369199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac7047-430d-4e24-a149-8d5a4c900b18 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.446388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cccf84-8bd1-420e-b0a8-148fe16bfa17 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:40.460385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:40.475396 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac7047-430d-4e24-a149-8d5a4c900b18 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:40.484370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b110f-321e-498a-afd4-20ea1566e11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:40.485110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b110f-321e-498a-afd4-20ea1566e11a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:40.485390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b110f-321e-498a-afd4-20ea1566e11a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:40.485660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b110f-321e-498a-afd4-20ea1566e11a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:40.494140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b110f-321e-498a-afd4-20ea1566e11a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:40.495637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b110f-321e-498a-afd4-20ea1566e11a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:40.503888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-951b110f-321e-498a-afd4-20ea1566e11a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:40.509406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:40.510274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:40.511147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 226/454] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:40.511695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b110f-321e-498a-afd4-20ea1566e11a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.519401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a17772-4a46-4d02-8918-32fe3eddc524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:40.519986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a17772-4a46-4d02-8918-32fe3eddc524 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:40.520265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a17772-4a46-4d02-8918-32fe3eddc524 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:40.520539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a17772-4a46-4d02-8918-32fe3eddc524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:40.531509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a17772-4a46-4d02-8918-32fe3eddc524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:40.532990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a17772-4a46-4d02-8918-32fe3eddc524 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-07a17772-4a46-4d02-8918-32fe3eddc524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:40.547959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:40.548828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:40.549721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:40.549721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:40.549721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:40.554239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 229/455] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:40.554830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a17772-4a46-4d02-8918-32fe3eddc524 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.641221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b110f-321e-498a-afd4-20ea1566e11a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:40.653501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a17772-4a46-4d02-8918-32fe3eddc524 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:40.677565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021f2c2-6692-499e-9bbb-398fefc7380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:40.678168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021f2c2-6692-499e-9bbb-398fefc7380a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:40.678449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021f2c2-6692-499e-9bbb-398fefc7380a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:40.678718 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021f2c2-6692-499e-9bbb-398fefc7380a None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:40.679371 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021f2c2-6692-499e-9bbb-398fefc7380a None admin] RBAC: Adding query filter params (name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:40.679671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021f2c2-6692-499e-9bbb-398fefc7380a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:40.681362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021f2c2-6692-499e-9bbb-398fefc7380a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:40.689613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecec09-4e45-44fe-83fb-2df81e82fc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:40.690248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecec09-4e45-44fe-83fb-2df81e82fc41 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:40.695115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecec09-4e45-44fe-83fb-2df81e82fc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:40.695231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 227/456] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:40.695309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecec09-4e45-44fe-83fb-2df81e82fc41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:40.695403 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021f2c2-6692-499e-9bbb-398fefc7380a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.705981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ecec09-4e45-44fe-83fb-2df81e82fc41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:40.707435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ecec09-4e45-44fe-83fb-2df81e82fc41 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-61ecec09-4e45-44fe-83fb-2df81e82fc41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:40.722886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:40.723830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:40.729487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:40.729487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:40.729487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:40.729487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 230/457] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:40.729487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecec09-4e45-44fe-83fb-2df81e82fc41 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.738757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:40.762416 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:40.813134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021f2c2-6692-499e-9bbb-398fefc7380a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:40.839267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecec09-4e45-44fe-83fb-2df81e82fc41 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:40.848393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6b59c-3aeb-4dc4-8f99-19b41180fae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:40.849067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6b59c-3aeb-4dc4-8f99-19b41180fae3 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:40.849356 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6b59c-3aeb-4dc4-8f99-19b41180fae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:40.849632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6b59c-3aeb-4dc4-8f99-19b41180fae3 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:40.850363 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6b59c-3aeb-4dc4-8f99-19b41180fae3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:40.850668 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6b59c-3aeb-4dc4-8f99-19b41180fae3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:40.852112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6b59c-3aeb-4dc4-8f99-19b41180fae3 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:40.873923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 228/458] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Oct 18 11:27:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:40.874646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6b59c-3aeb-4dc4-8f99-19b41180fae3 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.884679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531db13-bd49-4ffe-a9b0-2a0ffdd88788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:40.885348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531db13-bd49-4ffe-a9b0-2a0ffdd88788 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:40.885873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531db13-bd49-4ffe-a9b0-2a0ffdd88788 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:40.885873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531db13-bd49-4ffe-a9b0-2a0ffdd88788 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:40.886536 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531db13-bd49-4ffe-a9b0-2a0ffdd88788 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:40.886836 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531db13-bd49-4ffe-a9b0-2a0ffdd88788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:40.888203 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531db13-bd49-4ffe-a9b0-2a0ffdd88788 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:40.905468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 231/459] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:40.906278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531db13-bd49-4ffe-a9b0-2a0ffdd88788 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:40.972396 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:40.981871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6b59c-3aeb-4dc4-8f99-19b41180fae3 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.004434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:41.014302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531db13-bd49-4ffe-a9b0-2a0ffdd88788 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.020712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a96e6-9135-4b3b-9cb5-55dffe6dd8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.021515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a96e6-9135-4b3b-9cb5-55dffe6dd8fa None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.021949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a96e6-9135-4b3b-9cb5-55dffe6dd8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.022321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a96e6-9135-4b3b-9cb5-55dffe6dd8fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.033029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a96e6-9135-4b3b-9cb5-55dffe6dd8fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.034595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a96e6-9135-4b3b-9cb5-55dffe6dd8fa None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-126a96e6-9135-4b3b-9cb5-55dffe6dd8fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:41.049795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:41.050896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:41.051920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:41.051920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:41.051920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:41.054284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd90150-f27b-46e0-9537-2ae5f3731a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.054894 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd90150-f27b-46e0-9537-2ae5f3731a69 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.055165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd90150-f27b-46e0-9537-2ae5f3731a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.055433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd90150-f27b-46e0-9537-2ae5f3731a69 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.056045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 229/460] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:41.056175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd90150-f27b-46e0-9537-2ae5f3731a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:41.056372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd90150-f27b-46e0-9537-2ae5f3731a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.056947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a96e6-9135-4b3b-9cb5-55dffe6dd8fa None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.057762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd90150-f27b-46e0-9537-2ae5f3731a69 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.077480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 232/461] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:41.077480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd90150-f27b-46e0-9537-2ae5f3731a69 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.184028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a96e6-9135-4b3b-9cb5-55dffe6dd8fa None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.193730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd90150-f27b-46e0-9537-2ae5f3731a69 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.221471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67ca2d-94b6-420f-95ae-83ef88be4ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.222083 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67ca2d-94b6-420f-95ae-83ef88be4ebc None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.222359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67ca2d-94b6-420f-95ae-83ef88be4ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.222627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67ca2d-94b6-420f-95ae-83ef88be4ebc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.234641 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1449f4-aa5b-4659-8cd8-fcec4aa660ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.235664 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1449f4-aa5b-4659-8cd8-fcec4aa660ee None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.236214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1449f4-aa5b-4659-8cd8-fcec4aa660ee None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.236498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1449f4-aa5b-4659-8cd8-fcec4aa660ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.237777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:41.246349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1449f4-aa5b-4659-8cd8-fcec4aa660ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.248280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1449f4-aa5b-4659-8cd8-fcec4aa660ee None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-af1449f4-aa5b-4659-8cd8-fcec4aa660ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:41.263264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:41.264635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:41.265844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:41.265844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:41.265844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:41.265844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:41.265844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:41.265844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:41.265844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:41.265844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:41.265844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 233/462] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:41.266310 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1449f4-aa5b-4659-8cd8-fcec4aa660ee None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.286830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:41.300832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67ca2d-94b6-420f-95ae-83ef88be4ebc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.302476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67ca2d-94b6-420f-95ae-83ef88be4ebc None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.377479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1449f4-aa5b-4659-8cd8-fcec4aa660ee None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.415380 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 230/463] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:27:41.415380 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67ca2d-94b6-420f-95ae-83ef88be4ebc None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.418676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5208be-0c53-4833-8056-3a7a800e4ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.419481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5208be-0c53-4833-8056-3a7a800e4ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.419930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5208be-0c53-4833-8056-3a7a800e4ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.420349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5208be-0c53-4833-8056-3a7a800e4ec4 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.421036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5208be-0c53-4833-8056-3a7a800e4ec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:41.421338 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5208be-0c53-4833-8056-3a7a800e4ec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.422704 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5208be-0c53-4833-8056-3a7a800e4ec4 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.440033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 234/464] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:41.440033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5208be-0c53-4833-8056-3a7a800e4ec4 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.463007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:41.519200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:41.525599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67ca2d-94b6-420f-95ae-83ef88be4ebc None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.553332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5208be-0c53-4833-8056-3a7a800e4ec4 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.560975 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10718cdd-9a16-49bb-b723-a083a14a6096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.561668 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10718cdd-9a16-49bb-b723-a083a14a6096 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.561941 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10718cdd-9a16-49bb-b723-a083a14a6096 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.562209 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10718cdd-9a16-49bb-b723-a083a14a6096 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.570689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10718cdd-9a16-49bb-b723-a083a14a6096 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.572128 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10718cdd-9a16-49bb-b723-a083a14a6096 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-10718cdd-9a16-49bb-b723-a083a14a6096 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:41.584535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:41.585579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:41.586453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 231/465] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:41.587011 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10718cdd-9a16-49bb-b723-a083a14a6096 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.598383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478728b-a442-412b-877c-621a5ad2a6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.599022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478728b-a442-412b-877c-621a5ad2a6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.599298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478728b-a442-412b-877c-621a5ad2a6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.600009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478728b-a442-412b-877c-621a5ad2a6c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.600749 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478728b-a442-412b-877c-621a5ad2a6c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:41.601448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478728b-a442-412b-877c-621a5ad2a6c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.602987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478728b-a442-412b-877c-621a5ad2a6c5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.617647 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 235/466] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:41.618413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478728b-a442-412b-877c-621a5ad2a6c5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.708560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10718cdd-9a16-49bb-b723-a083a14a6096 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.711353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:41.724034 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478728b-a442-412b-877c-621a5ad2a6c5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.745187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edf72c-fc53-492b-83e2-501a078ca66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.745843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edf72c-fc53-492b-83e2-501a078ca66e None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.746122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edf72c-fc53-492b-83e2-501a078ca66e None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.746394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edf72c-fc53-492b-83e2-501a078ca66e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.755614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:41.758225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edf72c-fc53-492b-83e2-501a078ca66e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.759714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edf72c-fc53-492b-83e2-501a078ca66e None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.760999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef6c46-f213-4226-9dd0-6de2b128bd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.761644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef6c46-f213-4226-9dd0-6de2b128bd98 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.761916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef6c46-f213-4226-9dd0-6de2b128bd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.762182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef6c46-f213-4226-9dd0-6de2b128bd98 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.771100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef6c46-f213-4226-9dd0-6de2b128bd98 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.772564 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef6c46-f213-4226-9dd0-6de2b128bd98 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-60edf72c-fc53-492b-83e2-501a078ca66e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:41.774844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:41.775731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:41.776567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:41.776567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:41.776567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:41.776567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 232/467] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:41.776819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edf72c-fc53-492b-83e2-501a078ca66e None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-50ef6c46-f213-4226-9dd0-6de2b128bd98 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:41.787331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:41.788420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 236/468] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:41.789839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef6c46-f213-4226-9dd0-6de2b128bd98 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.882136 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edf72c-fc53-492b-83e2-501a078ca66e None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.889591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef6c46-f213-4226-9dd0-6de2b128bd98 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:41.915079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bd597-639a-49d4-9d75-32fcc27ef522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.915677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bd597-639a-49d4-9d75-32fcc27ef522 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.915948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bd597-639a-49d4-9d75-32fcc27ef522 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.916214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bd597-639a-49d4-9d75-32fcc27ef522 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.916952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bd597-639a-49d4-9d75-32fcc27ef522 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:41.917254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bd597-639a-49d4-9d75-32fcc27ef522 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.918687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bd597-639a-49d4-9d75-32fcc27ef522 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.924200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507e43a-58b5-4759-bfb9-76ce696961d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:41.924808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507e43a-58b5-4759-bfb9-76ce696961d2 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.925124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507e43a-58b5-4759-bfb9-76ce696961d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.925395 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507e43a-58b5-4759-bfb9-76ce696961d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.926116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507e43a-58b5-4759-bfb9-76ce696961d2 None admin] RBAC: Adding query filter params (role.id=f94c9d39cd6244b4b0a727e200ff2e94, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=87f8a4a8277745a9ad1976af9df31cb9) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:41.926438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507e43a-58b5-4759-bfb9-76ce696961d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:41.927836 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507e43a-58b5-4759-bfb9-76ce696961d2 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:41.933530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 237/469] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/role_assignments?user.id=87f8a4a8277745a9ad1976af9df31cb9&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=f94c9d39cd6244b4b0a727e200ff2e94 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:41.934696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520b3e-41a1-4551-82ad-8faad8aaeb5f None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:41.934978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520b3e-41a1-4551-82ad-8faad8aaeb5f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:41.935260 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520b3e-41a1-4551-82ad-8faad8aaeb5f None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:41.936627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 238/470] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:41.937222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520b3e-41a1-4551-82ad-8faad8aaeb5f None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.945636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 233/471] 10.222.0.17 () {56 vars in 1137 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:41.945636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bd597-639a-49d4-9d75-32fcc27ef522 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:41.963572 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:42.012081 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:42.059963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bd597-639a-49d4-9d75-32fcc27ef522 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:42.064932 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520b3e-41a1-4551-82ad-8faad8aaeb5f None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:42.097917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4834d8-6c53-43b1-8a6d-399ea485091a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:42.098544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4834d8-6c53-43b1-8a6d-399ea485091a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:42.098818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4834d8-6c53-43b1-8a6d-399ea485091a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:42.099088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4834d8-6c53-43b1-8a6d-399ea485091a None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:42.099831 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4834d8-6c53-43b1-8a6d-399ea485091a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:42.100135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4834d8-6c53-43b1-8a6d-399ea485091a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:42.101636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4834d8-6c53-43b1-8a6d-399ea485091a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:42.107663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc65002-a184-423b-915e-4dc8e1cfebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:42.108552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc65002-a184-423b-915e-4dc8e1cfebd5 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:42.109022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc65002-a184-423b-915e-4dc8e1cfebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:42.109301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc65002-a184-423b-915e-4dc8e1cfebd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:42.109987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc65002-a184-423b-915e-4dc8e1cfebd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:42.110565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc65002-a184-423b-915e-4dc8e1cfebd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:42.113001 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc65002-a184-423b-915e-4dc8e1cfebd5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:42.120101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 234/472] 10.222.0.17 () {56 vars in 1139 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:42.120983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4834d8-6c53-43b1-8a6d-399ea485091a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:42.127327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 239/473] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:42.128873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253faaf-0075-4662-9e8c-9fabe8735a37 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:42.129180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253faaf-0075-4662-9e8c-9fabe8735a37 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:42.129448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253faaf-0075-4662-9e8c-9fabe8735a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:42.147983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-0253faaf-0075-4662-9e8c-9fabe8735a37 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:42.149281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0253faaf-0075-4662-9e8c-9fabe8735a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:42.199481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:42.231379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 240/474] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:42.231989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253faaf-0075-4662-9e8c-9fabe8735a37 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:42.242161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4834d8-6c53-43b1-8a6d-399ea485091a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:42.280679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfc923-f8a8-4996-9d6d-c6e1d4ed75bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:42.281503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfc923-f8a8-4996-9d6d-c6e1d4ed75bf None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:42.281784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfc923-f8a8-4996-9d6d-c6e1d4ed75bf None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:42.282188 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfc923-f8a8-4996-9d6d-c6e1d4ed75bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:42.297126 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:42.358862 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253faaf-0075-4662-9e8c-9fabe8735a37 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:42.362022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfc923-f8a8-4996-9d6d-c6e1d4ed75bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:42.363484 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfc923-f8a8-4996-9d6d-c6e1d4ed75bf None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:42.401130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dbe8d-7412-4c37-b0f7-b29158e87f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:42.401746 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dbe8d-7412-4c37-b0f7-b29158e87f01 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:42.402062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dbe8d-7412-4c37-b0f7-b29158e87f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:42.402336 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dbe8d-7412-4c37-b0f7-b29158e87f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:42.442603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:42.480930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 235/475] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:27:42.481696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfc923-f8a8-4996-9d6d-c6e1d4ed75bf None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:42.488560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dbe8d-7412-4c37-b0f7-b29158e87f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:42.490598 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dbe8d-7412-4c37-b0f7-b29158e87f01 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:42.538363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:42.598679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 241/476] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:27:42.599825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dbe8d-7412-4c37-b0f7-b29158e87f01 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:42.608238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfc923-f8a8-4996-9d6d-c6e1d4ed75bf None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:42.644416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6da35a-6ff4-43bb-a5f2-5ca15a8a8c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:42.645071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6da35a-6ff4-43bb-a5f2-5ca15a8a8c94 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:42.645346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6da35a-6ff4-43bb-a5f2-5ca15a8a8c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:42.645613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6da35a-6ff4-43bb-a5f2-5ca15a8a8c94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:42.652358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6da35a-6ff4-43bb-a5f2-5ca15a8a8c94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:42.653820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6da35a-6ff4-43bb-a5f2-5ca15a8a8c94 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-7c6da35a-6ff4-43bb-a5f2-5ca15a8a8c94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:42.665276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:42.666134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:42.666963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:42.666963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:42.666963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:42.666963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:42.666963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:42.666963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:42.666963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:42.666963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:42.666963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 236/477] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:42.667410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6da35a-6ff4-43bb-a5f2-5ca15a8a8c94 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:42.704034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:42.709071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dbe8d-7412-4c37-b0f7-b29158e87f01 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:42.747136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec5abc-5941-423b-a51f-6189eab0d4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:42.749067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec5abc-5941-423b-a51f-6189eab0d4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:42.749359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec5abc-5941-423b-a51f-6189eab0d4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:42.749631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec5abc-5941-423b-a51f-6189eab0d4a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:42.760019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec5abc-5941-423b-a51f-6189eab0d4a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:42.761516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec5abc-5941-423b-a51f-6189eab0d4a1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:42.769511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-c9ec5abc-5941-423b-a51f-6189eab0d4a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:42.778473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:42.780115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:42.781614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:42.781614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:42.781614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:42.781614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 242/478] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:42.781614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec5abc-5941-423b-a51f-6189eab0d4a1 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:42.791401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6da35a-6ff4-43bb-a5f2-5ca15a8a8c94 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:42.833335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c89807-d459-4480-bc62-eb6bde15e7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:42.833967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c89807-d459-4480-bc62-eb6bde15e7ac None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:42.834242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c89807-d459-4480-bc62-eb6bde15e7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:42.834508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c89807-d459-4480-bc62-eb6bde15e7ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:42.845958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c89807-d459-4480-bc62-eb6bde15e7ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-01c89807-d459-4480-bc62-eb6bde15e7ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:42.851483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:42.852436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:42.853267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 237/479] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:42.853928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c89807-d459-4480-bc62-eb6bde15e7ac None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:42.900478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec5abc-5941-423b-a51f-6189eab0d4a1 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:42.940230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fceb12-55c5-4181-a1d3-34f479778276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:42.941159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fceb12-55c5-4181-a1d3-34f479778276 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:42.941441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fceb12-55c5-4181-a1d3-34f479778276 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:42.941709 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fceb12-55c5-4181-a1d3-34f479778276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:42.953748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fceb12-55c5-4181-a1d3-34f479778276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:42.955183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fceb12-55c5-4181-a1d3-34f479778276 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:42.955296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:42.965211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c89807-d459-4480-bc62-eb6bde15e7ac None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-66fceb12-55c5-4181-a1d3-34f479778276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:42.970638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:42.971710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:42.972621 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:42.972621 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:42.972621 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:42.973987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 243/480] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:42.974906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fceb12-55c5-4181-a1d3-34f479778276 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.006691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99b370-c733-4394-b7ba-c519cd33c7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.007220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b370-c733-4394-b7ba-c519cd33c7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.007489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b370-c733-4394-b7ba-c519cd33c7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.007749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b370-c733-4394-b7ba-c519cd33c7c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.008516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99b370-c733-4394-b7ba-c519cd33c7c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:43.008819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99b370-c733-4394-b7ba-c519cd33c7c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.010259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99b370-c733-4394-b7ba-c519cd33c7c0 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.024869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 238/481] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:43.025809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99b370-c733-4394-b7ba-c519cd33c7c0 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.031743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:43.088435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fceb12-55c5-4181-a1d3-34f479778276 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:43.122536 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f38471-273c-4e60-950c-74f3762bffac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.123840 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f38471-273c-4e60-950c-74f3762bffac None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.124045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f38471-273c-4e60-950c-74f3762bffac None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.124315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f38471-273c-4e60-950c-74f3762bffac None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.124976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f38471-273c-4e60-950c-74f3762bffac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:43.125283 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f38471-273c-4e60-950c-74f3762bffac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.126643 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f38471-273c-4e60-950c-74f3762bffac None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.134980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99b370-c733-4394-b7ba-c519cd33c7c0 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:43.139250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 244/482] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:43.139954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f38471-273c-4e60-950c-74f3762bffac None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.183521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a637d7-b26f-4735-9f5b-aa40ecfdda71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.184135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a637d7-b26f-4735-9f5b-aa40ecfdda71 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.184415 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a637d7-b26f-4735-9f5b-aa40ecfdda71 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.184686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a637d7-b26f-4735-9f5b-aa40ecfdda71 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.185441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a637d7-b26f-4735-9f5b-aa40ecfdda71 None admin] RBAC: Adding query filter params (name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:43.185743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a637d7-b26f-4735-9f5b-aa40ecfdda71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.187102 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a637d7-b26f-4735-9f5b-aa40ecfdda71 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.196983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 239/483] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:43.196983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a637d7-b26f-4735-9f5b-aa40ecfdda71 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.217163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:43.260429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f38471-273c-4e60-950c-74f3762bffac None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:43.272612 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:43.298599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f467c0a-f2e4-4595-81fa-4aca8c0a93bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.299218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f467c0a-f2e4-4595-81fa-4aca8c0a93bf None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.299484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f467c0a-f2e4-4595-81fa-4aca8c0a93bf None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.299734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f467c0a-f2e4-4595-81fa-4aca8c0a93bf None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.300375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f467c0a-f2e4-4595-81fa-4aca8c0a93bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:43.300679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f467c0a-f2e4-4595-81fa-4aca8c0a93bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.302109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f467c0a-f2e4-4595-81fa-4aca8c0a93bf None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.307628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a637d7-b26f-4735-9f5b-aa40ecfdda71 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:43.317794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 245/484] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:43.317794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f467c0a-f2e4-4595-81fa-4aca8c0a93bf None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.348705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd82e6e-9171-49ca-9265-390d4ca7ed6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.349344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd82e6e-9171-49ca-9265-390d4ca7ed6b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.349619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd82e6e-9171-49ca-9265-390d4ca7ed6b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.349884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd82e6e-9171-49ca-9265-390d4ca7ed6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.350608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd82e6e-9171-49ca-9265-390d4ca7ed6b None admin] RBAC: Adding query filter params (role.id=f94c9d39cd6244b4b0a727e200ff2e94, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=3d2a655f4d6f45039d641133e3b56592) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:43.350915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd82e6e-9171-49ca-9265-390d4ca7ed6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.352349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd82e6e-9171-49ca-9265-390d4ca7ed6b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.368903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 240/485] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/role_assignments?user.id=3d2a655f4d6f45039d641133e3b56592&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=f94c9d39cd6244b4b0a727e200ff2e94 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:43.368903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd82e6e-9171-49ca-9265-390d4ca7ed6b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.450221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f467c0a-f2e4-4595-81fa-4aca8c0a93bf None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:43.467327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:43.500078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd82e6e-9171-49ca-9265-390d4ca7ed6b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:43.506979 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2b707-b9bf-4ee5-a17e-722811446210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.507976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2b707-b9bf-4ee5-a17e-722811446210 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.508606 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2b707-b9bf-4ee5-a17e-722811446210 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.509828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2b707-b9bf-4ee5-a17e-722811446210 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.521200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2b707-b9bf-4ee5-a17e-722811446210 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.523019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2b707-b9bf-4ee5-a17e-722811446210 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-4cd2b707-b9bf-4ee5-a17e-722811446210 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:43.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:43.540678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:43.541538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:43.541538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:43.541538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:43.541538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:43.541538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:43.541538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:43.541538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:43.541538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:43.541538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 246/486] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:43.541989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2b707-b9bf-4ee5-a17e-722811446210 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.544156 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28114f7-f5ce-4e7b-87e8-402cc63bd07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.545613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28114f7-f5ce-4e7b-87e8-402cc63bd07e None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.546091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28114f7-f5ce-4e7b-87e8-402cc63bd07e None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.546552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28114f7-f5ce-4e7b-87e8-402cc63bd07e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.561241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28114f7-f5ce-4e7b-87e8-402cc63bd07e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.562691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28114f7-f5ce-4e7b-87e8-402cc63bd07e None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-d28114f7-f5ce-4e7b-87e8-402cc63bd07e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:43.581561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:43.582502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:43.583325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:43.583325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:43.583325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:43.583325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 241/487] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:43.583547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28114f7-f5ce-4e7b-87e8-402cc63bd07e None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.593883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:43.677975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2b707-b9bf-4ee5-a17e-722811446210 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:43.709944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28114f7-f5ce-4e7b-87e8-402cc63bd07e None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:43.724909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d468f-8c18-4faa-9390-e9d3eac15824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.726382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d468f-8c18-4faa-9390-e9d3eac15824 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.726941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d468f-8c18-4faa-9390-e9d3eac15824 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.727389 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d468f-8c18-4faa-9390-e9d3eac15824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.737359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d468f-8c18-4faa-9390-e9d3eac15824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.739933 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d468f-8c18-4faa-9390-e9d3eac15824 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.744270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-ac0d468f-8c18-4faa-9390-e9d3eac15824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:43.753159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:43.754032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:43.754857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:43.754857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:43.754857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:43.754857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:43.754857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:43.754857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:43.754857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:43.754857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:43.755265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7577c-8577-4af6-98ba-ee427691106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.755265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7577c-8577-4af6-98ba-ee427691106b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.755265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7577c-8577-4af6-98ba-ee427691106b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.755265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7577c-8577-4af6-98ba-ee427691106b None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.755530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7577c-8577-4af6-98ba-ee427691106b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:43.755530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7577c-8577-4af6-98ba-ee427691106b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.757141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7577c-8577-4af6-98ba-ee427691106b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.757939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 247/488] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:43.759140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d468f-8c18-4faa-9390-e9d3eac15824 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.770512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 242/489] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:43.775675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7577c-8577-4af6-98ba-ee427691106b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.881540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7577c-8577-4af6-98ba-ee427691106b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:43.895444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:43.909447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d468f-8c18-4faa-9390-e9d3eac15824 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:43.918124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b378bd9-2f7e-41e6-913f-353f29da566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.918410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b378bd9-2f7e-41e6-913f-353f29da566c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.918680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b378bd9-2f7e-41e6-913f-353f29da566c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.918947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b378bd9-2f7e-41e6-913f-353f29da566c None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.919689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b378bd9-2f7e-41e6-913f-353f29da566c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:43.920507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b378bd9-2f7e-41e6-913f-353f29da566c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.921962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b378bd9-2f7e-41e6-913f-353f29da566c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.934372 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 243/490] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:43.935317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b378bd9-2f7e-41e6-913f-353f29da566c None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.948623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bb97d-d4fa-4870-9ec4-bb2bbfa5f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:43.949568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bb97d-d4fa-4870-9ec4-bb2bbfa5f8ab None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:43.950067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bb97d-d4fa-4870-9ec4-bb2bbfa5f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:43.950548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bb97d-d4fa-4870-9ec4-bb2bbfa5f8ab None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:43.951444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bb97d-d4fa-4870-9ec4-bb2bbfa5f8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:43.952392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bb97d-d4fa-4870-9ec4-bb2bbfa5f8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:43.954077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bb97d-d4fa-4870-9ec4-bb2bbfa5f8ab None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:43.967514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 248/491] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:43.968678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bb97d-d4fa-4870-9ec4-bb2bbfa5f8ab None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:43.993769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:44.053262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b378bd9-2f7e-41e6-913f-353f29da566c None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:44.082330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bb97d-d4fa-4870-9ec4-bb2bbfa5f8ab None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:44.092289 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6a09c-d92c-417b-9f23-04fb30ea4955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:44.092918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6a09c-d92c-417b-9f23-04fb30ea4955 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:44.093199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6a09c-d92c-417b-9f23-04fb30ea4955 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:44.093464 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6a09c-d92c-417b-9f23-04fb30ea4955 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:44.094382 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6a09c-d92c-417b-9f23-04fb30ea4955 None admin] RBAC: Adding query filter params (role.id=f94c9d39cd6244b4b0a727e200ff2e94, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=bb051b00a16949798158bac1ac57d36f) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:44.094491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6a09c-d92c-417b-9f23-04fb30ea4955 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:44.096118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6a09c-d92c-417b-9f23-04fb30ea4955 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:44.108895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 244/492] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/role_assignments?user.id=bb051b00a16949798158bac1ac57d36f&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=f94c9d39cd6244b4b0a727e200ff2e94 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:44.108895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6a09c-d92c-417b-9f23-04fb30ea4955 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:44.143618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3825cea-6348-4946-9937-691636ed9476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:44.144242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3825cea-6348-4946-9937-691636ed9476 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:44.144516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3825cea-6348-4946-9937-691636ed9476 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:44.144795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3825cea-6348-4946-9937-691636ed9476 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:44.145557 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3825cea-6348-4946-9937-691636ed9476 None admin] RBAC: Adding query filter params (role.id=f94c9d39cd6244b4b0a727e200ff2e94, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=d301f3d9aeb644a38c31dbdbef2a48c7) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:44.145864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3825cea-6348-4946-9937-691636ed9476 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:44.147306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3825cea-6348-4946-9937-691636ed9476 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:44.162218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 249/493] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:43 2022] GET /identity/v3/role_assignments?user.id=d301f3d9aeb644a38c31dbdbef2a48c7&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=f94c9d39cd6244b4b0a727e200ff2e94 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:44.163715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929b183-7a34-4f07-b11e-74c776f93470 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:44.164000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929b183-7a34-4f07-b11e-74c776f93470 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:44.164271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929b183-7a34-4f07-b11e-74c776f93470 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:44.171252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 250/494] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:44.173624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457b56-8aa7-41ea-bb39-be892a2cddda None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:44.173904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457b56-8aa7-41ea-bb39-be892a2cddda None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:44.174492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457b56-8aa7-41ea-bb39-be892a2cddda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:44.221975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-52457b56-8aa7-41ea-bb39-be892a2cddda None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:44.223218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-52457b56-8aa7-41ea-bb39-be892a2cddda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:44.239844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:44.265236 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6a09c-d92c-417b-9f23-04fb30ea4955 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:44.282359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:44.306198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0e0f3-abfa-4869-84a9-09c1a9d091f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:44.306770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0e0f3-abfa-4869-84a9-09c1a9d091f6 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:44.307103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0e0f3-abfa-4869-84a9-09c1a9d091f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:44.307373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0e0f3-abfa-4869-84a9-09c1a9d091f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:44.325374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0e0f3-abfa-4869-84a9-09c1a9d091f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:44.328552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0e0f3-abfa-4869-84a9-09c1a9d091f6 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-96b0e0f3-abfa-4869-84a9-09c1a9d091f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:44.356159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:44.357226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:44.358274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 245/495] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:27:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:44.363716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0e0f3-abfa-4869-84a9-09c1a9d091f6 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:44.379988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 251/496] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:27:44.495098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0e0f3-abfa-4869-84a9-09c1a9d091f6 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:44.539420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a465c0-8ada-4ba8-9d75-92aba0514c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:44.540039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a465c0-8ada-4ba8-9d75-92aba0514c04 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:44.540321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a465c0-8ada-4ba8-9d75-92aba0514c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:44.540593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a465c0-8ada-4ba8-9d75-92aba0514c04 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:44.541356 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a465c0-8ada-4ba8-9d75-92aba0514c04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:44.541663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a465c0-8ada-4ba8-9d75-92aba0514c04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:44.543115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a465c0-8ada-4ba8-9d75-92aba0514c04 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:44.573320 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 246/497] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Oct 18 11:27:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:44.578820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52457b56-8aa7-41ea-bb39-be892a2cddda None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:44.705522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52457b56-8aa7-41ea-bb39-be892a2cddda None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:44.742330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef6c69-a5ca-4400-8476-ea3b8c4d3838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:44.742954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef6c69-a5ca-4400-8476-ea3b8c4d3838 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:44.743234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef6c69-a5ca-4400-8476-ea3b8c4d3838 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:44.743506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef6c69-a5ca-4400-8476-ea3b8c4d3838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:44.758936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:44.809450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a465c0-8ada-4ba8-9d75-92aba0514c04 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:44.818574 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef6c69-a5ca-4400-8476-ea3b8c4d3838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:44.820270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef6c69-a5ca-4400-8476-ea3b8c4d3838 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:44.864305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:44.940462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 252/498] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 18 11:27:44.955615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef6c69-a5ca-4400-8476-ea3b8c4d3838 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:44.979693 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a465c0-8ada-4ba8-9d75-92aba0514c04 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:45.045207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:45.048230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda00cd-21fd-443d-b9bc-769a71c53a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:45.048884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda00cd-21fd-443d-b9bc-769a71c53a04 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:45.049166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda00cd-21fd-443d-b9bc-769a71c53a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:45.049434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda00cd-21fd-443d-b9bc-769a71c53a04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:45.068040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda00cd-21fd-443d-b9bc-769a71c53a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-deda00cd-21fd-443d-b9bc-769a71c53a04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:45.072891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:45.073789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:45.074625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 247/499] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:27:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:45.082659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda00cd-21fd-443d-b9bc-769a71c53a04 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:45.163110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef6c69-a5ca-4400-8476-ea3b8c4d3838 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:45.228748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80575e0f-b177-4e2f-a0d9-8b8a53606ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:45.229662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80575e0f-b177-4e2f-a0d9-8b8a53606ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:45.230087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80575e0f-b177-4e2f-a0d9-8b8a53606ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:45.230506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80575e0f-b177-4e2f-a0d9-8b8a53606ac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:45.251141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80575e0f-b177-4e2f-a0d9-8b8a53606ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:45.252892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80575e0f-b177-4e2f-a0d9-8b8a53606ac1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:45.261225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-80575e0f-b177-4e2f-a0d9-8b8a53606ac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:45.279561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:45.280828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:45.282408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:45.282408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:45.282408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:45.284267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda00cd-21fd-443d-b9bc-769a71c53a04 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:45.285945 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 253/500] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:45.291966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80575e0f-b177-4e2f-a0d9-8b8a53606ac1 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:45.344694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be1cd8-7c1f-406b-a597-574942bc533c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:45.345342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be1cd8-7c1f-406b-a597-574942bc533c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:45.345620 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be1cd8-7c1f-406b-a597-574942bc533c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:45.345893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be1cd8-7c1f-406b-a597-574942bc533c None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:45.346562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be1cd8-7c1f-406b-a597-574942bc533c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:45.346863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be1cd8-7c1f-406b-a597-574942bc533c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:45.348509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be1cd8-7c1f-406b-a597-574942bc533c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:45.365098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 248/501] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:45.371145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be1cd8-7c1f-406b-a597-574942bc533c None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:45.483000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:45.494120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80575e0f-b177-4e2f-a0d9-8b8a53606ac1 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:45.558015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be1cd8-7c1f-406b-a597-574942bc533c None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:45.561468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df028f2e-d9a1-4890-97e1-87c45c4a92fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:45.562084 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df028f2e-d9a1-4890-97e1-87c45c4a92fc None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:45.562360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df028f2e-d9a1-4890-97e1-87c45c4a92fc None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:45.562630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df028f2e-d9a1-4890-97e1-87c45c4a92fc None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:45.563287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df028f2e-d9a1-4890-97e1-87c45c4a92fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:45.563588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df028f2e-d9a1-4890-97e1-87c45c4a92fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:45.565004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df028f2e-d9a1-4890-97e1-87c45c4a92fc None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:45.587081 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 254/502] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:45.593792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df028f2e-d9a1-4890-97e1-87c45c4a92fc None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:45.631409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb26263-bf7f-4c8c-870d-7ed16b5c8727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:45.632606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb26263-bf7f-4c8c-870d-7ed16b5c8727 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:45.633061 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb26263-bf7f-4c8c-870d-7ed16b5c8727 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:45.633482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb26263-bf7f-4c8c-870d-7ed16b5c8727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:45.653716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb26263-bf7f-4c8c-870d-7ed16b5c8727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:45.655230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb26263-bf7f-4c8c-870d-7ed16b5c8727 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-dcb26263-bf7f-4c8c-870d-7ed16b5c8727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:45.682262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:45.683439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:45.684514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:45.684514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:45.684514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:45.684907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 249/503] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:45.692332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb26263-bf7f-4c8c-870d-7ed16b5c8727 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:45.710330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:45.742191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df028f2e-d9a1-4890-97e1-87c45c4a92fc None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:45.812348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d15fe-4b5c-471e-b208-332ad4788640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:45.813318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d15fe-4b5c-471e-b208-332ad4788640 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:45.813602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d15fe-4b5c-471e-b208-332ad4788640 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:45.813868 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d15fe-4b5c-471e-b208-332ad4788640 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:45.831141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d15fe-4b5c-471e-b208-332ad4788640 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:45.832621 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d15fe-4b5c-471e-b208-332ad4788640 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-653d15fe-4b5c-471e-b208-332ad4788640 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:45.852539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:45.853546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:45.854533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:45.854533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:45.854533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:45.854533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:45.854533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:45.854533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:45.854533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:45.854533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:45.859524 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 255/504] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:45.860657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d15fe-4b5c-471e-b208-332ad4788640 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:45.881343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:45.884262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb26263-bf7f-4c8c-870d-7ed16b5c8727 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:45.950333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba50fa4-e095-40cf-af5e-14947b83349c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:45.951342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba50fa4-e095-40cf-af5e-14947b83349c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:45.951644 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba50fa4-e095-40cf-af5e-14947b83349c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:45.951916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba50fa4-e095-40cf-af5e-14947b83349c None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:45.952720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba50fa4-e095-40cf-af5e-14947b83349c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:45.953237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba50fa4-e095-40cf-af5e-14947b83349c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:45.954722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba50fa4-e095-40cf-af5e-14947b83349c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:45.973387 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 250/505] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:45.975672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5eee1-a0e0-476c-8e27-2d00ac71c2e2 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:45.976026 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5eee1-a0e0-476c-8e27-2d00ac71c2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:45.976306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5eee1-a0e0-476c-8e27-2d00ac71c2e2 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:45.977996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 251/506] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:45.978755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5eee1-a0e0-476c-8e27-2d00ac71c2e2 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:46.006277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d15fe-4b5c-471e-b208-332ad4788640 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:46.052091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6da21-9354-4397-89b8-03093f32e5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:46.053165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6da21-9354-4397-89b8-03093f32e5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.053591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6da21-9354-4397-89b8-03093f32e5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.053986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6da21-9354-4397-89b8-03093f32e5e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.054931 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6da21-9354-4397-89b8-03093f32e5e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:46.055249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6da21-9354-4397-89b8-03093f32e5e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:46.057439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6da21-9354-4397-89b8-03093f32e5e1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:46.062315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:46.077386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 256/507] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:46.078811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e50ce-8305-4aba-b429-ad53f4e51b20 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.079153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e50ce-8305-4aba-b429-ad53f4e51b20 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.079436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e50ce-8305-4aba-b429-ad53f4e51b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.101830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-6c2e50ce-8305-4aba-b429-ad53f4e51b20 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:46.103098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6c2e50ce-8305-4aba-b429-ad53f4e51b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:46.113259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5eee1-a0e0-476c-8e27-2d00ac71c2e2 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:46.162313 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415417b-a476-457c-b823-79cbe74b2de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:46.162966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415417b-a476-457c-b823-79cbe74b2de3 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.163269 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415417b-a476-457c-b823-79cbe74b2de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.167057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415417b-a476-457c-b823-79cbe74b2de3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.179864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415417b-a476-457c-b823-79cbe74b2de3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:46.181348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415417b-a476-457c-b823-79cbe74b2de3 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:46.195138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 257/508] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-5415417b-a476-457c-b823-79cbe74b2de3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:46.195393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:46.196293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:46.197293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:46.197293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:46.197293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:46.197293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:46.197293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:46.197293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:46.197293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:46.197293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:46.197293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:27:46.197293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:46.199237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e50ce-8305-4aba-b429-ad53f4e51b20 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:46.204984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 252/509] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Oct 18 11:27:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:46.204984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415417b-a476-457c-b823-79cbe74b2de3 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:46.218791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:46.387130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415417b-a476-457c-b823-79cbe74b2de3 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:46.391706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e50ce-8305-4aba-b429-ad53f4e51b20 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:46.436062 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:46.449666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d338c5c-dcf9-4032-acd2-bd7769d2a3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:46.450273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d338c5c-dcf9-4032-acd2-bd7769d2a3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.450551 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d338c5c-dcf9-4032-acd2-bd7769d2a3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.450852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d338c5c-dcf9-4032-acd2-bd7769d2a3c0 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.451596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d338c5c-dcf9-4032-acd2-bd7769d2a3c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:46.451894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d338c5c-dcf9-4032-acd2-bd7769d2a3c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:46.455212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d338c5c-dcf9-4032-acd2-bd7769d2a3c0 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:46.458704 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d999706-f3f6-448b-9e3f-3070d5ac32c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:46.459311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d999706-f3f6-448b-9e3f-3070d5ac32c3 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.459582 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d999706-f3f6-448b-9e3f-3070d5ac32c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.459845 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d999706-f3f6-448b-9e3f-3070d5ac32c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.460569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d999706-f3f6-448b-9e3f-3070d5ac32c3 None admin] RBAC: Adding query filter params (role.id=f94c9d39cd6244b4b0a727e200ff2e94, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=4be9de8292214872a59c1466f7513539) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:46.460886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d999706-f3f6-448b-9e3f-3070d5ac32c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:46.462313 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d999706-f3f6-448b-9e3f-3070d5ac32c3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:46.475104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 258/510] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:46 2022] GET /identity/v3/role_assignments?user.id=4be9de8292214872a59c1466f7513539&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=f94c9d39cd6244b4b0a727e200ff2e94 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:46.477093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d999706-f3f6-448b-9e3f-3070d5ac32c3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:46.498446 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 253/511] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Oct 18 11:27:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:46.499563 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d338c5c-dcf9-4032-acd2-bd7769d2a3c0 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:46.613261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:46.681057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d999706-f3f6-448b-9e3f-3070d5ac32c3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:46.718560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d338c5c-dcf9-4032-acd2-bd7769d2a3c0 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:46.737200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2e948-8a0d-420b-b096-0d2023e299e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:46.738003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2e948-8a0d-420b-b096-0d2023e299e8 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.738405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2e948-8a0d-420b-b096-0d2023e299e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.739669 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2e948-8a0d-420b-b096-0d2023e299e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.753530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2e948-8a0d-420b-b096-0d2023e299e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:46.755570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2e948-8a0d-420b-b096-0d2023e299e8 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-b9e2e948-8a0d-420b-b096-0d2023e299e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:46.771738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:46.773290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:46.774715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:46.774715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:46.774715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:46.774715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 259/512] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:46.774715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be8e98-7882-49c9-9c61-242e7dea60c2 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.774715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be8e98-7882-49c9-9c61-242e7dea60c2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.774715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be8e98-7882-49c9-9c61-242e7dea60c2 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.776170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 260/513] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:46.779187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be8e98-7882-49c9-9c61-242e7dea60c2 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:46.787294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7056f0-6c3b-4e59-9b47-60036be44d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:46.788087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7056f0-6c3b-4e59-9b47-60036be44d61 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.788448 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7056f0-6c3b-4e59-9b47-60036be44d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.788858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7056f0-6c3b-4e59-9b47-60036be44d61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.805087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7056f0-6c3b-4e59-9b47-60036be44d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:46.807410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7056f0-6c3b-4e59-9b47-60036be44d61 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-0a7056f0-6c3b-4e59-9b47-60036be44d61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:46.828326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:46.829405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:46.830280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:46.830280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:46.830280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:46.830280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 254/514] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:46.831306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ea34-d3b7-46f8-95c9-f0d9b2f85ce2 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.831582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ea34-d3b7-46f8-95c9-f0d9b2f85ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.831855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ea34-d3b7-46f8-95c9-f0d9b2f85ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.854075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c768ea34-d3b7-46f8-95c9-f0d9b2f85ce2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:46.855264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c768ea34-d3b7-46f8-95c9-f0d9b2f85ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:46.888256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:46.911490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be8e98-7882-49c9-9c61-242e7dea60c2 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:46.932974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 255/515] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:46.932974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260edc8-3882-4622-b876-9296c24c7a10 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.932974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260edc8-3882-4622-b876-9296c24c7a10 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.932974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260edc8-3882-4622-b876-9296c24c7a10 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.933862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 256/516] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:46.934457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260edc8-3882-4622-b876-9296c24c7a10 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:46.940173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:46.953581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd126484-1143-41bd-a913-b3bc4cd1172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:46.954562 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd126484-1143-41bd-a913-b3bc4cd1172d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.955035 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd126484-1143-41bd-a913-b3bc4cd1172d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.955486 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd126484-1143-41bd-a913-b3bc4cd1172d None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:46.956457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd126484-1143-41bd-a913-b3bc4cd1172d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:46.956968 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd126484-1143-41bd-a913-b3bc4cd1172d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:46.958581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd126484-1143-41bd-a913-b3bc4cd1172d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:46.974761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 261/517] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:46.977661 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d84d1-26b6-45e0-97ae-256bc86d0162 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:46.978991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d84d1-26b6-45e0-97ae-256bc86d0162 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:46.979467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d84d1-26b6-45e0-97ae-256bc86d0162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:47.008332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-539d84d1-26b6-45e0-97ae-256bc86d0162 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:47.010297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-539d84d1-26b6-45e0-97ae-256bc86d0162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:47.073360 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260edc8-3882-4622-b876-9296c24c7a10 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:47.093280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 262/518] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:47.094612 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d84d1-26b6-45e0-97ae-256bc86d0162 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:47.122155 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa56b62-d134-47ed-99c4-30f19946d86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:47.122824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa56b62-d134-47ed-99c4-30f19946d86a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:47.123099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa56b62-d134-47ed-99c4-30f19946d86a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:47.123366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa56b62-d134-47ed-99c4-30f19946d86a None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:47.124025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa56b62-d134-47ed-99c4-30f19946d86a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:47.124322 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa56b62-d134-47ed-99c4-30f19946d86a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:47.125803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa56b62-d134-47ed-99c4-30f19946d86a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:47.150285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 257/519] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:47.156731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa56b62-d134-47ed-99c4-30f19946d86a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:47.223423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:47.251555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d84d1-26b6-45e0-97ae-256bc86d0162 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:47.265884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:47.288924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6167d-ea32-4e9c-b6d4-f9b47662f225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:47.289540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6167d-ea32-4e9c-b6d4-f9b47662f225 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:47.289809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6167d-ea32-4e9c-b6d4-f9b47662f225 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:47.290077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6167d-ea32-4e9c-b6d4-f9b47662f225 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:47.290716 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6167d-ea32-4e9c-b6d4-f9b47662f225 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:47.292153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6167d-ea32-4e9c-b6d4-f9b47662f225 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:47.313969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa56b62-d134-47ed-99c4-30f19946d86a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:47.317380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 263/520] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:27:47 2022] POST /identity/v3/users => generated 254 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:27:47.318263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6167d-ea32-4e9c-b6d4-f9b47662f225 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:47.402944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4ba79-e554-49d6-a462-09d2852b45f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:47.403701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4ba79-e554-49d6-a462-09d2852b45f4 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:47.403945 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4ba79-e554-49d6-a462-09d2852b45f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:47.404717 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4ba79-e554-49d6-a462-09d2852b45f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:47.418714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4ba79-e554-49d6-a462-09d2852b45f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:47.421560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4ba79-e554-49d6-a462-09d2852b45f4 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-52a4ba79-e554-49d6-a462-09d2852b45f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:47.438913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:47.439908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:47.440807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:47.440807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:47.440807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:47.440807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:47.440807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:47.440807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:47.440807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:47.440807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:47.444502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 258/521] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:47.445915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4ba79-e554-49d6-a462-09d2852b45f4 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:47.457949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6167d-ea32-4e9c-b6d4-f9b47662f225 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:47.506643 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a6c06-51b0-40bb-8b68-47fea1c75d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:47.507353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a6c06-51b0-40bb-8b68-47fea1c75d29 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:47.507636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a6c06-51b0-40bb-8b68-47fea1c75d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:47.507909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a6c06-51b0-40bb-8b68-47fea1c75d29 None admin] PATH_INFO: `/v3/services/network` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:47.508526 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a6c06-51b0-40bb-8b68-47fea1c75d29 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:47.510005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a6c06-51b0-40bb-8b68-47fea1c75d29 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-b09a6c06-51b0-40bb-8b68-47fea1c75d29 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:47.526856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:47.527935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 18 11:27:47.528802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:47.532311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 264/522] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:47 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:47.533886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a6c06-51b0-40bb-8b68-47fea1c75d29 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:47.570319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:47.578465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:47.600462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4ba79-e554-49d6-a462-09d2852b45f4 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:47.639473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5680bc-435b-44a6-b143-66298e0f5a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:47.640154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5680bc-435b-44a6-b143-66298e0f5a97 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:47.640432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5680bc-435b-44a6-b143-66298e0f5a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:47.640700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5680bc-435b-44a6-b143-66298e0f5a97 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:47.641314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5680bc-435b-44a6-b143-66298e0f5a97 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:47.642670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5680bc-435b-44a6-b143-66298e0f5a97 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:47.651230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a6c06-51b0-40bb-8b68-47fea1c75d29 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-ab5680bc-435b-44a6-b143-66298e0f5a97 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:47.654877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:47.655865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:47.656707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 259/523] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Oct 18 11:27:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:47.658165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5680bc-435b-44a6-b143-66298e0f5a97 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:47.693886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d4017-a53a-4541-ba46-bb2a62afcd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:47.694611 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d4017-a53a-4541-ba46-bb2a62afcd10 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:47.694892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d4017-a53a-4541-ba46-bb2a62afcd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:47.695165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d4017-a53a-4541-ba46-bb2a62afcd10 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:47.695862 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d4017-a53a-4541-ba46-bb2a62afcd10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:47.696168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d4017-a53a-4541-ba46-bb2a62afcd10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:47.697662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d4017-a53a-4541-ba46-bb2a62afcd10 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:47.710302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 265/524] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:47.711227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d4017-a53a-4541-ba46-bb2a62afcd10 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:47.764009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5680bc-435b-44a6-b143-66298e0f5a97 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:47.801278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3baba0-fd2c-40f9-bcce-72a4e039bac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:47.801895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3baba0-fd2c-40f9-bcce-72a4e039bac4 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:47.802173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3baba0-fd2c-40f9-bcce-72a4e039bac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:47.802442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3baba0-fd2c-40f9-bcce-72a4e039bac4 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:47.803075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3baba0-fd2c-40f9-bcce-72a4e039bac4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:47.803376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3baba0-fd2c-40f9-bcce-72a4e039bac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:47.804749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3baba0-fd2c-40f9-bcce-72a4e039bac4 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:47.811702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:47.814455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 260/525] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:27:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:47.815318 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3baba0-fd2c-40f9-bcce-72a4e039bac4 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:47.823542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d4017-a53a-4541-ba46-bb2a62afcd10 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:47.832244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:47.861014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5cdb6-c677-42e3-8907-9bbbd6b162c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:47.861879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5cdb6-c677-42e3-8907-9bbbd6b162c0 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:47.862309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5cdb6-c677-42e3-8907-9bbbd6b162c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:47.862757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5cdb6-c677-42e3-8907-9bbbd6b162c0 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:47.863545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5cdb6-c677-42e3-8907-9bbbd6b162c0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:47.864250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5cdb6-c677-42e3-8907-9bbbd6b162c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:47.865868 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5cdb6-c677-42e3-8907-9bbbd6b162c0 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:47.875633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 266/526] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Oct 18 11:27:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:47.876769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5cdb6-c677-42e3-8907-9bbbd6b162c0 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:47.927216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3baba0-fd2c-40f9-bcce-72a4e039bac4 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:47.965857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98902bf-a4e6-45cc-ab6e-170d00588d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:47.967092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98902bf-a4e6-45cc-ab6e-170d00588d52 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:47.967373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98902bf-a4e6-45cc-ab6e-170d00588d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:47.967645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98902bf-a4e6-45cc-ab6e-170d00588d52 None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/87f8a4a8277745a9ad1976af9df31cb9/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:47.969832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98902bf-a4e6-45cc-ab6e-170d00588d52 None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=87f8a4a8277745a9ad1976af9df31cb9, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:47.971304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98902bf-a4e6-45cc-ab6e-170d00588d52 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:47.989255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 261/527] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:27:47 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/87f8a4a8277745a9ad1976af9df31cb9/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:47.990271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98902bf-a4e6-45cc-ab6e-170d00588d52 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:47.992560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5cdb6-c677-42e3-8907-9bbbd6b162c0 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:48.031187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4f0a4-1b29-4087-8689-8bc171b9cadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:48.032175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4f0a4-1b29-4087-8689-8bc171b9cadb None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:48.032663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4f0a4-1b29-4087-8689-8bc171b9cadb None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:48.033291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4f0a4-1b29-4087-8689-8bc171b9cadb None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:48.034148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4f0a4-1b29-4087-8689-8bc171b9cadb None admin] RBAC: Adding query filter params (type=network) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:48.034658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4f0a4-1b29-4087-8689-8bc171b9cadb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:48.036282 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4f0a4-1b29-4087-8689-8bc171b9cadb None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:48.043407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 267/528] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:27:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:48.087973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:48.124327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98902bf-a4e6-45cc-ab6e-170d00588d52 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:48.174128 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a2b13-7e85-4c72-83bb-490e3a0cefab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:48.174755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a2b13-7e85-4c72-83bb-490e3a0cefab None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:48.175031 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a2b13-7e85-4c72-83bb-490e3a0cefab None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:48.175301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a2b13-7e85-4c72-83bb-490e3a0cefab None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:48.178489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a2b13-7e85-4c72-83bb-490e3a0cefab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:48.178489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a2b13-7e85-4c72-83bb-490e3a0cefab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:48.178489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a2b13-7e85-4c72-83bb-490e3a0cefab None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:48.187529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 262/529] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Oct 18 11:27:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:48.813165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f15f9-f8c6-4603-a0c2-b0f2c1f59380 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:48.813675 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f15f9-f8c6-4603-a0c2-b0f2c1f59380 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:48.813753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f15f9-f8c6-4603-a0c2-b0f2c1f59380 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:48.815419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 268/530] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:48.820906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51815b21-601c-43dd-a9f8-b9a0fcad893f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:48.821381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51815b21-601c-43dd-a9f8-b9a0fcad893f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:48.821463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51815b21-601c-43dd-a9f8-b9a0fcad893f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:48.840604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-51815b21-601c-43dd-a9f8-b9a0fcad893f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:48.842389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-51815b21-601c-43dd-a9f8-b9a0fcad893f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:48.928822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 263/531] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:49.241030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f15f9-f8c6-4603-a0c2-b0f2c1f59380 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:49.275988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:49.398524 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f15f9-f8c6-4603-a0c2-b0f2c1f59380 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:49.436174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d669c4f-e74e-4486-bd8a-5810d1a9fa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:49.436743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d669c4f-e74e-4486-bd8a-5810d1a9fa86 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:49.437055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d669c4f-e74e-4486-bd8a-5810d1a9fa86 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:49.437327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d669c4f-e74e-4486-bd8a-5810d1a9fa86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:49.448660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d669c4f-e74e-4486-bd8a-5810d1a9fa86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-9d669c4f-e74e-4486-bd8a-5810d1a9fa86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:49.453805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:49.454817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:49.455662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 269/532] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:27:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:49.461195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51815b21-601c-43dd-a9f8-b9a0fcad893f None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:49.537623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:49.573240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51815b21-601c-43dd-a9f8-b9a0fcad893f None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:49.609306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5267c2-04b3-4ea3-9e40-3b92cc47001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:49.609942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5267c2-04b3-4ea3-9e40-3b92cc47001a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:49.610220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5267c2-04b3-4ea3-9e40-3b92cc47001a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:49.610493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5267c2-04b3-4ea3-9e40-3b92cc47001a None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:49.611177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5267c2-04b3-4ea3-9e40-3b92cc47001a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:49.611477 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5267c2-04b3-4ea3-9e40-3b92cc47001a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:49.612951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5267c2-04b3-4ea3-9e40-3b92cc47001a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:49.623554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 264/533] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:49.629201 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d669c4f-e74e-4486-bd8a-5810d1a9fa86 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:49.722829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:49.746124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d669c4f-e74e-4486-bd8a-5810d1a9fa86 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:49.800200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58c31b-1a93-4ff0-a879-8234b15f0f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:49.800200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c31b-1a93-4ff0-a879-8234b15f0f9b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:49.800200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c31b-1a93-4ff0-a879-8234b15f0f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:49.800200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c31b-1a93-4ff0-a879-8234b15f0f9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:49.811104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58c31b-1a93-4ff0-a879-8234b15f0f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:49.812865 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58c31b-1a93-4ff0-a879-8234b15f0f9b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-8b58c31b-1a93-4ff0-a879-8234b15f0f9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:49.832651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:49.838114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:49.839000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:49.839000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:49.839000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:49.840505 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 270/534] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:49.847018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5267c2-04b3-4ea3-9e40-3b92cc47001a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:49.991435 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5267c2-04b3-4ea3-9e40-3b92cc47001a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:50.021285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:50.027231 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d966524-3c8c-4ee7-89b5-c7f2c9eabca8 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:50.027865 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d966524-3c8c-4ee7-89b5-c7f2c9eabca8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:50.028330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d966524-3c8c-4ee7-89b5-c7f2c9eabca8 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:50.029872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab41495-e736-46e0-b19e-55138b3db00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:50.030495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab41495-e736-46e0-b19e-55138b3db00b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:50.030779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab41495-e736-46e0-b19e-55138b3db00b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:50.031121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab41495-e736-46e0-b19e-55138b3db00b None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:50.031555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 271/535] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:50.031740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab41495-e736-46e0-b19e-55138b3db00b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:50.032008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab41495-e736-46e0-b19e-55138b3db00b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:50.036634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab41495-e736-46e0-b19e-55138b3db00b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:50.038606 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b652fb0-3887-4bfc-992d-ff8f6f380441 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:50.039098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b652fb0-3887-4bfc-992d-ff8f6f380441 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:50.039558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b652fb0-3887-4bfc-992d-ff8f6f380441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:50.052660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 265/536] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:50.053632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab41495-e736-46e0-b19e-55138b3db00b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:50.060158 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7b652fb0-3887-4bfc-992d-ff8f6f380441 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:50.061709 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7b652fb0-3887-4bfc-992d-ff8f6f380441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:50.187014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 272/537] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 18 11:27:50.205491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab41495-e736-46e0-b19e-55138b3db00b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:50.255110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e76c76-8dc2-49c9-9717-88a5f9b387bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:50.255680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76c76-8dc2-49c9-9717-88a5f9b387bc None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:50.255954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76c76-8dc2-49c9-9717-88a5f9b387bc None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:50.256222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76c76-8dc2-49c9-9717-88a5f9b387bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:50.265090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e76c76-8dc2-49c9-9717-88a5f9b387bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:50.266561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e76c76-8dc2-49c9-9717-88a5f9b387bc None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-26e76c76-8dc2-49c9-9717-88a5f9b387bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:50.283744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:50.284833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:50.285819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:50.285819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:50.285819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:50.285819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:50.285819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:50.285819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:50.285819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:50.285819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:50.285819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 18 11:27:50.285819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:50.288087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 266/538] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:27:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:50.293685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b652fb0-3887-4bfc-992d-ff8f6f380441 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:50.385632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:50.440337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b652fb0-3887-4bfc-992d-ff8f6f380441 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:50.490517 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf9454-c3c5-4fe1-8f8c-2acca31a707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:50.491524 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf9454-c3c5-4fe1-8f8c-2acca31a707d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:50.492009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf9454-c3c5-4fe1-8f8c-2acca31a707d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:50.492481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf9454-c3c5-4fe1-8f8c-2acca31a707d None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:50.493466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf9454-c3c5-4fe1-8f8c-2acca31a707d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:50.493978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf9454-c3c5-4fe1-8f8c-2acca31a707d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:50.495718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf9454-c3c5-4fe1-8f8c-2acca31a707d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:50.522335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 273/539] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Oct 18 11:27:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:50.528013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e76c76-8dc2-49c9-9717-88a5f9b387bc None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:50.590725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf9454-c3c5-4fe1-8f8c-2acca31a707d None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:50.610024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:50.690961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e76c76-8dc2-49c9-9717-88a5f9b387bc None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:50.765594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f781b-3d48-49f6-8f04-4a42ee4cbff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:50.766629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f781b-3d48-49f6-8f04-4a42ee4cbff3 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:50.767174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f781b-3d48-49f6-8f04-4a42ee4cbff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:50.767650 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f781b-3d48-49f6-8f04-4a42ee4cbff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:50.776619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf9454-c3c5-4fe1-8f8c-2acca31a707d None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:50.789042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f781b-3d48-49f6-8f04-4a42ee4cbff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:50.790996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f781b-3d48-49f6-8f04-4a42ee4cbff3 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:50.804000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-f53f781b-3d48-49f6-8f04-4a42ee4cbff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:50.810037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:50.811568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:50.812457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:50.812457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:50.812457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:50.814000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 267/540] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:50.815402 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3717af-be07-44a5-9e82-febb863fd2b0 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:50.815787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3717af-be07-44a5-9e82-febb863fd2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:50.816060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3717af-be07-44a5-9e82-febb863fd2b0 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:50.820774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f323d-0b84-4f74-97d9-f93364f1d2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:50.821520 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f323d-0b84-4f74-97d9-f93364f1d2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:50.821634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 268/541] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:50.821796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f323d-0b84-4f74-97d9-f93364f1d2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:50.822062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f323d-0b84-4f74-97d9-f93364f1d2c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:50.822561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3717af-be07-44a5-9e82-febb863fd2b0 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:50.835939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f323d-0b84-4f74-97d9-f93364f1d2c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-742f323d-0b84-4f74-97d9-f93364f1d2c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:50.842670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:50.844779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 274/542] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:50.846822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80709095-3047-4476-bebc-561c78ca1d61 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:50.848571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80709095-3047-4476-bebc-561c78ca1d61 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:50.848571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80709095-3047-4476-bebc-561c78ca1d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:50.868249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-80709095-3047-4476-bebc-561c78ca1d61 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:50.869683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-80709095-3047-4476-bebc-561c78ca1d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:50.944974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 275/543] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:50.945936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80709095-3047-4476-bebc-561c78ca1d61 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:50.960819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3717af-be07-44a5-9e82-febb863fd2b0 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:50.973211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:51.032899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d0203-5745-489e-9727-8a3b64f1c358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:51.033444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d0203-5745-489e-9727-8a3b64f1c358 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.033728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d0203-5745-489e-9727-8a3b64f1c358 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.034003 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d0203-5745-489e-9727-8a3b64f1c358 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.034662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d0203-5745-489e-9727-8a3b64f1c358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:51.034968 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d0203-5745-489e-9727-8a3b64f1c358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:51.036343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d0203-5745-489e-9727-8a3b64f1c358 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:51.065201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 269/544] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:51.066508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3851e4c-06fc-4677-bb24-ed0e30cd4256 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.066798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3851e4c-06fc-4677-bb24-ed0e30cd4256 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.067073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3851e4c-06fc-4677-bb24-ed0e30cd4256 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.068546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 270/545] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:51.072463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3851e4c-06fc-4677-bb24-ed0e30cd4256 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:51.095925 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80709095-3047-4476-bebc-561c78ca1d61 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:51.110694 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:51.147728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d623b6-c421-4a95-8f8a-e1cd88cf59fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:51.148357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d623b6-c421-4a95-8f8a-e1cd88cf59fe None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.148777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d623b6-c421-4a95-8f8a-e1cd88cf59fe None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.149217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d623b6-c421-4a95-8f8a-e1cd88cf59fe None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.149900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d623b6-c421-4a95-8f8a-e1cd88cf59fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:51.150461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d623b6-c421-4a95-8f8a-e1cd88cf59fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:51.152651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d623b6-c421-4a95-8f8a-e1cd88cf59fe None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:51.167147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 276/546] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:51.167147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c5db3-ec73-4632-b069-7c4a6e4d2780 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.167147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c5db3-ec73-4632-b069-7c4a6e4d2780 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.171273 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c5db3-ec73-4632-b069-7c4a6e4d2780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.195548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3851e4c-06fc-4677-bb24-ed0e30cd4256 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:51.200629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-942c5db3-ec73-4632-b069-7c4a6e4d2780 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:51.201890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-942c5db3-ec73-4632-b069-7c4a6e4d2780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:51.236146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc751d49-32bd-47ed-ae3b-57ba32e6c8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:51.236935 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc751d49-32bd-47ed-ae3b-57ba32e6c8ac None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.237205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc751d49-32bd-47ed-ae3b-57ba32e6c8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.237472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc751d49-32bd-47ed-ae3b-57ba32e6c8ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.245296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc751d49-32bd-47ed-ae3b-57ba32e6c8ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:51.246925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc751d49-32bd-47ed-ae3b-57ba32e6c8ac None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-fc751d49-32bd-47ed-ae3b-57ba32e6c8ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:51.259280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:51.261424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:51.263855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:51.263855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:51.263855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:51.263855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:51.263855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:51.263855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:51.263855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:51.263855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:51.263855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 271/547] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:51.263855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc751d49-32bd-47ed-ae3b-57ba32e6c8ac None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:51.283380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 277/548] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:51.284635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebd4a7-67aa-4f2b-ad91-1fdd3d8a9034 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.284962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebd4a7-67aa-4f2b-ad91-1fdd3d8a9034 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.285232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebd4a7-67aa-4f2b-ad91-1fdd3d8a9034 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.286458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 278/549] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:51.287147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebd4a7-67aa-4f2b-ad91-1fdd3d8a9034 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:51.298325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:51.375623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc751d49-32bd-47ed-ae3b-57ba32e6c8ac None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:51.403394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebd4a7-67aa-4f2b-ad91-1fdd3d8a9034 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:51.405747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:51.410671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99804e7a-d979-427c-bb0b-44c48cd47f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:51.411306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99804e7a-d979-427c-bb0b-44c48cd47f45 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.411580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99804e7a-d979-427c-bb0b-44c48cd47f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.411846 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99804e7a-d979-427c-bb0b-44c48cd47f45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.423580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99804e7a-d979-427c-bb0b-44c48cd47f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:51.425194 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99804e7a-d979-427c-bb0b-44c48cd47f45 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-99804e7a-d979-427c-bb0b-44c48cd47f45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:51.440406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:51.441487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:51.442388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:51.442388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:51.442388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:51.442388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 272/550] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:51.442388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99804e7a-d979-427c-bb0b-44c48cd47f45 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:51.449339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b912d9f-c785-4f2f-9f58-e8a08bd2f066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:51.449969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b912d9f-c785-4f2f-9f58-e8a08bd2f066 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.450240 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b912d9f-c785-4f2f-9f58-e8a08bd2f066 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.450502 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b912d9f-c785-4f2f-9f58-e8a08bd2f066 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.451231 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b912d9f-c785-4f2f-9f58-e8a08bd2f066 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:51.451528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b912d9f-c785-4f2f-9f58-e8a08bd2f066 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:51.452982 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b912d9f-c785-4f2f-9f58-e8a08bd2f066 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:51.465440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 279/551] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:51.467580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b49f2-7152-4123-aaf0-cb98bbc4c2d0 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.468521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b49f2-7152-4123-aaf0-cb98bbc4c2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.469056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b49f2-7152-4123-aaf0-cb98bbc4c2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.488259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e12b49f2-7152-4123-aaf0-cb98bbc4c2d0 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:51.490109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e12b49f2-7152-4123-aaf0-cb98bbc4c2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:51.527243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:51.556921 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99804e7a-d979-427c-bb0b-44c48cd47f45 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:51.562626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 280/552] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 18 11:27:51.563493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b49f2-7152-4123-aaf0-cb98bbc4c2d0 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:51.594024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f478a5-5ba0-4107-a830-aa7dcb74e4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:51.594649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f478a5-5ba0-4107-a830-aa7dcb74e4e6 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.594930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f478a5-5ba0-4107-a830-aa7dcb74e4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.595198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f478a5-5ba0-4107-a830-aa7dcb74e4e6 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.595861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f478a5-5ba0-4107-a830-aa7dcb74e4e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:51.597460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f478a5-5ba0-4107-a830-aa7dcb74e4e6 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:51.615179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 273/553] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:27:51 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:27:51.616596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f478a5-5ba0-4107-a830-aa7dcb74e4e6 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:51.697760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:51.711295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b49f2-7152-4123-aaf0-cb98bbc4c2d0 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:51.761568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f478a5-5ba0-4107-a830-aa7dcb74e4e6 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:51.768112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c689818-c5b6-47af-9524-027b68720038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:51.768752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c689818-c5b6-47af-9524-027b68720038 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.769060 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c689818-c5b6-47af-9524-027b68720038 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.769335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c689818-c5b6-47af-9524-027b68720038 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.769999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c689818-c5b6-47af-9524-027b68720038 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:51.770302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c689818-c5b6-47af-9524-027b68720038 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:51.771853 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c689818-c5b6-47af-9524-027b68720038 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:51.792879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 281/554] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:51.792879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c689818-c5b6-47af-9524-027b68720038 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:51.810897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:51.820006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468cfe4-70d6-42ce-b297-d42f49552751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:51.820724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468cfe4-70d6-42ce-b297-d42f49552751 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.821508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468cfe4-70d6-42ce-b297-d42f49552751 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.822146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468cfe4-70d6-42ce-b297-d42f49552751 None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/4be9de8292214872a59c1466f7513539/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.824594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468cfe4-70d6-42ce-b297-d42f49552751 None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=4be9de8292214872a59c1466f7513539, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:51.826787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468cfe4-70d6-42ce-b297-d42f49552751 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:51.842102 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 274/555] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:27:51 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/4be9de8292214872a59c1466f7513539/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:27:51.842963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468cfe4-70d6-42ce-b297-d42f49552751 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:51.939222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c689818-c5b6-47af-9524-027b68720038 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:51.983642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468cfe4-70d6-42ce-b297-d42f49552751 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:51.992199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269e5a5-8d1c-47f5-b008-fb91377792a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:51.993190 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269e5a5-8d1c-47f5-b008-fb91377792a5 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:51.993619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269e5a5-8d1c-47f5-b008-fb91377792a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:51.994130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269e5a5-8d1c-47f5-b008-fb91377792a5 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:51.994950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269e5a5-8d1c-47f5-b008-fb91377792a5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:51.997309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269e5a5-8d1c-47f5-b008-fb91377792a5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:52.024273 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 282/556] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:27:51 2022] POST /identity/v3/services => generated 251 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:27:52.025874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269e5a5-8d1c-47f5-b008-fb91377792a5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:52.036271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545627ff-3685-4cdd-a4ad-fb5eb155a48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:52.036953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545627ff-3685-4cdd-a4ad-fb5eb155a48c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:52.037291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545627ff-3685-4cdd-a4ad-fb5eb155a48c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:52.037566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545627ff-3685-4cdd-a4ad-fb5eb155a48c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:52.042761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:52.047974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545627ff-3685-4cdd-a4ad-fb5eb155a48c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:52.049634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545627ff-3685-4cdd-a4ad-fb5eb155a48c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-545627ff-3685-4cdd-a4ad-fb5eb155a48c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:52.066596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:52.067697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:52.068749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 275/557] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Oct 18 11:27:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:52.075768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545627ff-3685-4cdd-a4ad-fb5eb155a48c None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:52.146984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:52.191442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269e5a5-8d1c-47f5-b008-fb91377792a5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:52.225409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545627ff-3685-4cdd-a4ad-fb5eb155a48c None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:52.249738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7c1bc-dbf8-45c0-90e1-dd0a91635390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:52.250399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7c1bc-dbf8-45c0-90e1-dd0a91635390 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:52.250680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7c1bc-dbf8-45c0-90e1-dd0a91635390 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:52.251112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7c1bc-dbf8-45c0-90e1-dd0a91635390 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:52.269310 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7c1bc-dbf8-45c0-90e1-dd0a91635390 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-1cb7c1bc-dbf8-45c0-90e1-dd0a91635390 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:52.274645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:52.275592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:52.276412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:52.281163 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 283/558] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:27:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:52.281442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74122e61-a900-4869-b3bd-58f3a8b1835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:52.282061 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74122e61-a900-4869-b3bd-58f3a8b1835c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:52.282339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74122e61-a900-4869-b3bd-58f3a8b1835c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:52.282610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74122e61-a900-4869-b3bd-58f3a8b1835c None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:52.283171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7c1bc-dbf8-45c0-90e1-dd0a91635390 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:52.283356 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74122e61-a900-4869-b3bd-58f3a8b1835c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:52.283644 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74122e61-a900-4869-b3bd-58f3a8b1835c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:52.286070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74122e61-a900-4869-b3bd-58f3a8b1835c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:52.307016 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 276/559] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Oct 18 11:27:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:52.315581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74122e61-a900-4869-b3bd-58f3a8b1835c None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:52.385224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:52.423738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7c1bc-dbf8-45c0-90e1-dd0a91635390 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:52.469288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74122e61-a900-4869-b3bd-58f3a8b1835c None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:52.476118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:52.507013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e27d6-fef8-48b0-a45f-e70941fa6bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:52.507690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e27d6-fef8-48b0-a45f-e70941fa6bec None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:52.507980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e27d6-fef8-48b0-a45f-e70941fa6bec None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:52.508252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e27d6-fef8-48b0-a45f-e70941fa6bec None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:52.508924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e27d6-fef8-48b0-a45f-e70941fa6bec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:52.509229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e27d6-fef8-48b0-a45f-e70941fa6bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:52.510682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e27d6-fef8-48b0-a45f-e70941fa6bec None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:52.528239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 284/560] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:52.541644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e27d6-fef8-48b0-a45f-e70941fa6bec None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:52.553358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c2165-c873-4c4c-a7a6-60b76e45c1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:52.554030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2165-c873-4c4c-a7a6-60b76e45c1af None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:52.554315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2165-c873-4c4c-a7a6-60b76e45c1af None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:52.554582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2165-c873-4c4c-a7a6-60b76e45c1af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:52.574992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c2165-c873-4c4c-a7a6-60b76e45c1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:52.577040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c2165-c873-4c4c-a7a6-60b76e45c1af None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-148c2165-c873-4c4c-a7a6-60b76e45c1af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:52.603681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:52.604676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:52.605616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:52.605616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:52.605616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:52.610015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 277/561] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:52.610641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c2165-c873-4c4c-a7a6-60b76e45c1af None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:52.737401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e27d6-fef8-48b0-a45f-e70941fa6bec None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:52.801287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7646c-5406-42e2-8495-7a1ddf8c1606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:52.802555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7646c-5406-42e2-8495-7a1ddf8c1606 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:52.803064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7646c-5406-42e2-8495-7a1ddf8c1606 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:52.803570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7646c-5406-42e2-8495-7a1ddf8c1606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:52.821273 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:52.825027 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7646c-5406-42e2-8495-7a1ddf8c1606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:52.828185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7646c-5406-42e2-8495-7a1ddf8c1606 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-39f7646c-5406-42e2-8495-7a1ddf8c1606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:52.855398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:52.856394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:52.857347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:52.857347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:52.857347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:52.859335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 285/562] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:52.867930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c2165-c873-4c4c-a7a6-60b76e45c1af None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:52.868613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7646c-5406-42e2-8495-7a1ddf8c1606 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:52.914405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac6cef-49a7-4968-8b39-6bc34a4273b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:52.915275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6cef-49a7-4968-8b39-6bc34a4273b9 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:52.915863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6cef-49a7-4968-8b39-6bc34a4273b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:52.917056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6cef-49a7-4968-8b39-6bc34a4273b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:52.918047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac6cef-49a7-4968-8b39-6bc34a4273b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:52.919670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac6cef-49a7-4968-8b39-6bc34a4273b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:52.921421 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac6cef-49a7-4968-8b39-6bc34a4273b9 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:52.944493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 278/563] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:52.949758 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac6cef-49a7-4968-8b39-6bc34a4273b9 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:52.985393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:53.042283 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7646c-5406-42e2-8495-7a1ddf8c1606 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:53.115320 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eea596-4062-4213-8b29-c3f285f49efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:53.117502 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eea596-4062-4213-8b29-c3f285f49efc None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:53.118137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eea596-4062-4213-8b29-c3f285f49efc None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:53.118720 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eea596-4062-4213-8b29-c3f285f49efc None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:53.120195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eea596-4062-4213-8b29-c3f285f49efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:53.120869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eea596-4062-4213-8b29-c3f285f49efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:53.125764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eea596-4062-4213-8b29-c3f285f49efc None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:53.144930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 286/564] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:53.150972 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eea596-4062-4213-8b29-c3f285f49efc None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:53.151497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac6cef-49a7-4968-8b39-6bc34a4273b9 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:53.230168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003faa8-bdeb-4942-97cc-2c9e24d61383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:53.231127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003faa8-bdeb-4942-97cc-2c9e24d61383 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:53.231581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003faa8-bdeb-4942-97cc-2c9e24d61383 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:53.234146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003faa8-bdeb-4942-97cc-2c9e24d61383 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:53.238907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:53.249293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003faa8-bdeb-4942-97cc-2c9e24d61383 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:53.251130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003faa8-bdeb-4942-97cc-2c9e24d61383 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-8003faa8-bdeb-4942-97cc-2c9e24d61383 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:53.275613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:53.276736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:53.277866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:53.277866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:53.277866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:53.277866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:53.277866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:53.277866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:53.277866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:53.277866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:53.283846 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 279/565] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:53.287202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003faa8-bdeb-4942-97cc-2c9e24d61383 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:53.345901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eea596-4062-4213-8b29-c3f285f49efc None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:53.403080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:53.429002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc70a2-7997-495c-96d2-8dd6a17df9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:53.429678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc70a2-7997-495c-96d2-8dd6a17df9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:53.429960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc70a2-7997-495c-96d2-8dd6a17df9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:53.430237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc70a2-7997-495c-96d2-8dd6a17df9c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:53.430617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003faa8-bdeb-4942-97cc-2c9e24d61383 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:53.449636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc70a2-7997-495c-96d2-8dd6a17df9c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:53.451133 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc70a2-7997-495c-96d2-8dd6a17df9c7 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-5ddc70a2-7997-495c-96d2-8dd6a17df9c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:53.464521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:53.465534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:53.467760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:53.467760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:53.467760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:53.467760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:53.467760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:53.467760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:53.467760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:53.467760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:53.467760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:27:53.467760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:53.471251 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 287/566] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Oct 18 11:27:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:53.472224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc70a2-7997-495c-96d2-8dd6a17df9c7 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:53.475307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117db5e-c28c-436c-bfbe-92f2ceb7478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:53.477624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117db5e-c28c-436c-bfbe-92f2ceb7478c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:53.477895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117db5e-c28c-436c-bfbe-92f2ceb7478c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:53.478176 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117db5e-c28c-436c-bfbe-92f2ceb7478c None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:53.478884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117db5e-c28c-436c-bfbe-92f2ceb7478c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:53.479192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117db5e-c28c-436c-bfbe-92f2ceb7478c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:53.480635 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117db5e-c28c-436c-bfbe-92f2ceb7478c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:53.504530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 280/567] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:53.510129 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117db5e-c28c-436c-bfbe-92f2ceb7478c None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:53.590105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc70a2-7997-495c-96d2-8dd6a17df9c7 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:53.593924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:53.634887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f8bdb-670f-42fe-88ea-57165422012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:53.635529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f8bdb-670f-42fe-88ea-57165422012c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:53.635839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f8bdb-670f-42fe-88ea-57165422012c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:53.636083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f8bdb-670f-42fe-88ea-57165422012c None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:53.640998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f8bdb-670f-42fe-88ea-57165422012c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:53.640998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f8bdb-670f-42fe-88ea-57165422012c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:53.640998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f8bdb-670f-42fe-88ea-57165422012c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:53.658901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 288/568] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Oct 18 11:27:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:53.663959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f8bdb-670f-42fe-88ea-57165422012c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:53.765664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117db5e-c28c-436c-bfbe-92f2ceb7478c None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:53.830592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f8bdb-670f-42fe-88ea-57165422012c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:53.845065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d026989-e9bd-4903-a258-9b2701f0c1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:53.846905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d026989-e9bd-4903-a258-9b2701f0c1ba None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:53.847718 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d026989-e9bd-4903-a258-9b2701f0c1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:53.848465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d026989-e9bd-4903-a258-9b2701f0c1ba None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/74136098746a4c10b1a74051b75bbd2b/roles/f94c9d39cd6244b4b0a727e200ff2e94` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:53.866686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:53.868320 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c95500-fd4e-40e7-b276-bcf003bb7840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:53.869166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c95500-fd4e-40e7-b276-bcf003bb7840 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:53.869722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c95500-fd4e-40e7-b276-bcf003bb7840 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:53.870013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c95500-fd4e-40e7-b276-bcf003bb7840 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:53.902145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:53.918318 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d026989-e9bd-4903-a258-9b2701f0c1ba None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=74136098746a4c10b1a74051b75bbd2b, role_id=f94c9d39cd6244b4b0a727e200ff2e94)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:53.920202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d026989-e9bd-4903-a258-9b2701f0c1ba None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:53.938418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 281/569] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:27:53 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/74136098746a4c10b1a74051b75bbd2b/roles/f94c9d39cd6244b4b0a727e200ff2e94 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:27:53.974447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c95500-fd4e-40e7-b276-bcf003bb7840 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:53.976380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c95500-fd4e-40e7-b276-bcf003bb7840 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:54.121508 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 289/570] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 18 11:27:54.127126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d026989-e9bd-4903-a258-9b2701f0c1ba None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:54.261837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d026989-e9bd-4903-a258-9b2701f0c1ba None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:54.303631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3997a-2aeb-4cd5-8f47-be4ca53ce172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:54.304316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3997a-2aeb-4cd5-8f47-be4ca53ce172 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:54.304598 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3997a-2aeb-4cd5-8f47-be4ca53ce172 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:54.304894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3997a-2aeb-4cd5-8f47-be4ca53ce172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:54.310541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb737b-d5a1-4aba-a0fe-04f03cb80d5c None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:54.310825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb737b-d5a1-4aba-a0fe-04f03cb80d5c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:54.311097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb737b-d5a1-4aba-a0fe-04f03cb80d5c None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:54.312669 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 290/571] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:54.318535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3997a-2aeb-4cd5-8f47-be4ca53ce172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:54.319354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3233797-c4ff-45c3-b07f-016b95c52345 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:54.319636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3233797-c4ff-45c3-b07f-016b95c52345 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:54.319903 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3233797-c4ff-45c3-b07f-016b95c52345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:54.322127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3997a-2aeb-4cd5-8f47-be4ca53ce172 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:54.339590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-f3233797-c4ff-45c3-b07f-016b95c52345 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:54.340866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f3233797-c4ff-45c3-b07f-016b95c52345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-63e3997a-2aeb-4cd5-8f47-be4ca53ce172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:54.344445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:54.345482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:54.346370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:54.346370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:54.346370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:54.346370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 282/572] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:54.351621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3997a-2aeb-4cd5-8f47-be4ca53ce172 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:54.406811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:54.481843 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:54.504023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 291/573] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:27:54.526861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3997a-2aeb-4cd5-8f47-be4ca53ce172 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:54.586915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e069420-0022-4aa2-81c1-19327f31d020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:54.587533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e069420-0022-4aa2-81c1-19327f31d020 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:54.587811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e069420-0022-4aa2-81c1-19327f31d020 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:54.588084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e069420-0022-4aa2-81c1-19327f31d020 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:54.588737 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e069420-0022-4aa2-81c1-19327f31d020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:54.589065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e069420-0022-4aa2-81c1-19327f31d020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:54.590440 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e069420-0022-4aa2-81c1-19327f31d020 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:54.614525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 283/574] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:54.619971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3233797-c4ff-45c3-b07f-016b95c52345 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:54.670301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf8d2f-4a4a-4bec-aa1b-f3e17dab25c7 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:54.670662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf8d2f-4a4a-4bec-aa1b-f3e17dab25c7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:54.670873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf8d2f-4a4a-4bec-aa1b-f3e17dab25c7 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:54.672472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 284/575] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:54.678237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116e55e-90e5-45cb-919c-683981b3c382 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:54.678519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116e55e-90e5-45cb-919c-683981b3c382 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:54.678792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116e55e-90e5-45cb-919c-683981b3c382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:54.712594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-8116e55e-90e5-45cb-919c-683981b3c382 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:54.713873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8116e55e-90e5-45cb-919c-683981b3c382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:54.804171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3233797-c4ff-45c3-b07f-016b95c52345 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:54.823961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 285/576] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:27:54.824914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116e55e-90e5-45cb-919c-683981b3c382 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:54.863367 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecfae8-fff2-4b50-9be9-73697a292a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:54.864279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecfae8-fff2-4b50-9be9-73697a292a70 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:54.864558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecfae8-fff2-4b50-9be9-73697a292a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:54.864828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecfae8-fff2-4b50-9be9-73697a292a70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:54.876137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ecfae8-fff2-4b50-9be9-73697a292a70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:54.877208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:54.877687 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ecfae8-fff2-4b50-9be9-73697a292a70 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-c5ecfae8-fff2-4b50-9be9-73697a292a70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:54.906007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:54.906975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:54.907859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:54.907859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:54.907859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:54.907859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:54.907859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:54.907859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:54.907859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:54.907859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:54.907859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 292/577] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:54.914632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecfae8-fff2-4b50-9be9-73697a292a70 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:54.990801 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116e55e-90e5-45cb-919c-683981b3c382 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:55.014991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:55.036678 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861922dd-8f04-4f25-9230-3e1990fd27fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:55.037358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861922dd-8f04-4f25-9230-3e1990fd27fe None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.037636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861922dd-8f04-4f25-9230-3e1990fd27fe None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.037907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861922dd-8f04-4f25-9230-3e1990fd27fe None admin] PATH_INFO: `/v3/services/network` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.038613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861922dd-8f04-4f25-9230-3e1990fd27fe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:55.040113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861922dd-8f04-4f25-9230-3e1990fd27fe None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-861922dd-8f04-4f25-9230-3e1990fd27fe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:55.049617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:55.050821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:55.051851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 286/578] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:54 2022] GET /identity/v3/services/network => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:55.053122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef48a08-25ac-43d4-a738-8979c3dda926 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.053122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef48a08-25ac-43d4-a738-8979c3dda926 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.053122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef48a08-25ac-43d4-a738-8979c3dda926 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.058089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 287/579] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:55.059007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef48a08-25ac-43d4-a738-8979c3dda926 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:55.078534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecfae8-fff2-4b50-9be9-73697a292a70 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:55.118750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b664c7b-d832-4386-8e6c-874ea0a0db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:55.119371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b664c7b-d832-4386-8e6c-874ea0a0db5f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.119917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b664c7b-d832-4386-8e6c-874ea0a0db5f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.120193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b664c7b-d832-4386-8e6c-874ea0a0db5f None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.120926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b664c7b-d832-4386-8e6c-874ea0a0db5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:55.121245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b664c7b-d832-4386-8e6c-874ea0a0db5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:55.123665 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b664c7b-d832-4386-8e6c-874ea0a0db5f None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:55.136882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 293/580] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:55.142879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64814979-e0f6-45b3-806c-039134cc47cb None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.143209 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64814979-e0f6-45b3-806c-039134cc47cb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.143727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64814979-e0f6-45b3-806c-039134cc47cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.164272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-64814979-e0f6-45b3-806c-039134cc47cb None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:55.166041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-64814979-e0f6-45b3-806c-039134cc47cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:55.178497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:55.190739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef48a08-25ac-43d4-a738-8979c3dda926 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:55.230296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b30d1-534b-4115-8c4d-36b051a82fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:55.231249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b30d1-534b-4115-8c4d-36b051a82fec None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.231532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b30d1-534b-4115-8c4d-36b051a82fec None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.233321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b30d1-534b-4115-8c4d-36b051a82fec None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.233321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b30d1-534b-4115-8c4d-36b051a82fec None admin] RBAC: Adding query filter params (name=network) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:55.233321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b30d1-534b-4115-8c4d-36b051a82fec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:55.234680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b30d1-534b-4115-8c4d-36b051a82fec None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:55.239726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 288/581] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:27:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:55.240506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b30d1-534b-4115-8c4d-36b051a82fec None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:55.246816 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 294/582] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:27:55.248466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64814979-e0f6-45b3-806c-039134cc47cb None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:55.302110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:55.363202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b30d1-534b-4115-8c4d-36b051a82fec None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:55.368902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64814979-e0f6-45b3-806c-039134cc47cb None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:55.407079 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b5b19-85bd-4d83-bfc3-7bd1b7d71262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:55.407696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b5b19-85bd-4d83-bfc3-7bd1b7d71262 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.407973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b5b19-85bd-4d83-bfc3-7bd1b7d71262 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.408243 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b5b19-85bd-4d83-bfc3-7bd1b7d71262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.409014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b5b19-85bd-4d83-bfc3-7bd1b7d71262 None admin] RBAC: Adding query filter params (role.id=8d9ff9da6ad946a6bc49336777182ff9, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=87f8a4a8277745a9ad1976af9df31cb9) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:55.409324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b5b19-85bd-4d83-bfc3-7bd1b7d71262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:55.409977 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b64ddb-b7d2-4836-8d23-d513ae954c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:55.410751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b5b19-85bd-4d83-bfc3-7bd1b7d71262 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:55.411053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b64ddb-b7d2-4836-8d23-d513ae954c58 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.411524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b64ddb-b7d2-4836-8d23-d513ae954c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.412045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b64ddb-b7d2-4836-8d23-d513ae954c58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.420983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 295/583] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:55 2022] GET /identity/v3/role_assignments?user.id=87f8a4a8277745a9ad1976af9df31cb9&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=8d9ff9da6ad946a6bc49336777182ff9 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:55.420983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b5b19-85bd-4d83-bfc3-7bd1b7d71262 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:55.441005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b64ddb-b7d2-4836-8d23-d513ae954c58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-66b64ddb-b7d2-4836-8d23-d513ae954c58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:55.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:55.453870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:55.454802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 289/584] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:27:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:55.465581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b64ddb-b7d2-4836-8d23-d513ae954c58 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:55.507050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:55.583995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b5b19-85bd-4d83-bfc3-7bd1b7d71262 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:55.588421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:55.629955 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b64ddb-b7d2-4836-8d23-d513ae954c58 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:55.633927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc6fb2-3a19-4bf4-88d9-c049b5b3ff66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:55.634542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc6fb2-3a19-4bf4-88d9-c049b5b3ff66 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.634819 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc6fb2-3a19-4bf4-88d9-c049b5b3ff66 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.635088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc6fb2-3a19-4bf4-88d9-c049b5b3ff66 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.635719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc6fb2-3a19-4bf4-88d9-c049b5b3ff66 None admin] RBAC: Adding query filter params (type=network) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:55.636019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc6fb2-3a19-4bf4-88d9-c049b5b3ff66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:55.637464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc6fb2-3a19-4bf4-88d9-c049b5b3ff66 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:55.644724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 296/585] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:27:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:55.645688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc6fb2-3a19-4bf4-88d9-c049b5b3ff66 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:55.672378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e3c7-40d4-4438-9aa6-b572dfb7f15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:55.673578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e3c7-40d4-4438-9aa6-b572dfb7f15f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.673863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e3c7-40d4-4438-9aa6-b572dfb7f15f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.674137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e3c7-40d4-4438-9aa6-b572dfb7f15f None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.674799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311e3c7-40d4-4438-9aa6-b572dfb7f15f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:55.675099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311e3c7-40d4-4438-9aa6-b572dfb7f15f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:55.676971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311e3c7-40d4-4438-9aa6-b572dfb7f15f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:55.688383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 290/586] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:27:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:55.689165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e3c7-40d4-4438-9aa6-b572dfb7f15f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:55.770101 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc6fb2-3a19-4bf4-88d9-c049b5b3ff66 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:55.788229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:55.809127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbb652-e0c8-4203-9051-b1a180702d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:55.810184 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbb652-e0c8-4203-9051-b1a180702d90 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.810687 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbb652-e0c8-4203-9051-b1a180702d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.811172 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbb652-e0c8-4203-9051-b1a180702d90 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.811985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbb652-e0c8-4203-9051-b1a180702d90 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:55.813589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbb652-e0c8-4203-9051-b1a180702d90 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:55.815317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e3c7-40d4-4438-9aa6-b572dfb7f15f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-19dbb652-e0c8-4203-9051-b1a180702d90 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:55.823221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:55.824246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 18 11:27:55.825136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:55.827939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 297/587] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Oct 18 11:27:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:55.829020 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbb652-e0c8-4203-9051-b1a180702d90 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:55.858377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c3a2a-a92c-4581-9339-bba201e6e96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:55.858995 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c3a2a-a92c-4581-9339-bba201e6e96c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.859294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c3a2a-a92c-4581-9339-bba201e6e96c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.859596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c3a2a-a92c-4581-9339-bba201e6e96c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.860291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c3a2a-a92c-4581-9339-bba201e6e96c None admin] RBAC: Adding query filter params (interface=public, service_id=2787631bfc6d4a7fb9623da0986e6bca, region_id=RegionOne) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:55.860639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c3a2a-a92c-4581-9339-bba201e6e96c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:55.862048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c3a2a-a92c-4581-9339-bba201e6e96c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:55.873038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 291/588] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Oct 18 11:27:55 2022] GET /identity/v3/endpoints?service_id=2787631bfc6d4a7fb9623da0986e6bca&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:55.874072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c3a2a-a92c-4581-9339-bba201e6e96c None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:55.875378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:55.949189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbb652-e0c8-4203-9051-b1a180702d90 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:55.985803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfafdb-27bc-4da6-a350-bd7c6800d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:55.986466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfafdb-27bc-4da6-a350-bd7c6800d57e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:55.986846 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfafdb-27bc-4da6-a350-bd7c6800d57e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:55.987222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfafdb-27bc-4da6-a350-bd7c6800d57e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:55.998600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfafdb-27bc-4da6-a350-bd7c6800d57e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:56.000188 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c3a2a-a92c-4581-9339-bba201e6e96c None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:56.000324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfafdb-27bc-4da6-a350-bd7c6800d57e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-87cfafdb-27bc-4da6-a350-bd7c6800d57e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:56.016719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:56.018307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:56.019427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:56.019427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:56.019427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:56.020618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 298/589] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:56.021871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfafdb-27bc-4da6-a350-bd7c6800d57e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:56.052742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85448f6b-7c80-4b15-8f58-4528ce64e593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:56.053813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85448f6b-7c80-4b15-8f58-4528ce64e593 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:56.054248 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85448f6b-7c80-4b15-8f58-4528ce64e593 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:56.054704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85448f6b-7c80-4b15-8f58-4528ce64e593 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:56.055483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85448f6b-7c80-4b15-8f58-4528ce64e593 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:56.056187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85448f6b-7c80-4b15-8f58-4528ce64e593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:56.058025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85448f6b-7c80-4b15-8f58-4528ce64e593 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:56.064328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 292/590] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Oct 18 11:27:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:56.065639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85448f6b-7c80-4b15-8f58-4528ce64e593 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:56.077152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:56.124004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:56.149555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfafdb-27bc-4da6-a350-bd7c6800d57e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:56.195632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035ed36-94e6-4e1a-9d85-fd7bc36d6a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:56.196997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035ed36-94e6-4e1a-9d85-fd7bc36d6a91 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:56.197752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035ed36-94e6-4e1a-9d85-fd7bc36d6a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:56.198208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035ed36-94e6-4e1a-9d85-fd7bc36d6a91 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:56.199317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035ed36-94e6-4e1a-9d85-fd7bc36d6a91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:56.202101 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035ed36-94e6-4e1a-9d85-fd7bc36d6a91 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:56.208523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85448f6b-7c80-4b15-8f58-4528ce64e593 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:56.209860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 299/591] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Oct 18 11:27:56 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:56.212163 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035ed36-94e6-4e1a-9d85-fd7bc36d6a91 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:56.333697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bc6ef-193d-4521-a3c1-cd0d8dba924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:56.334332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bc6ef-193d-4521-a3c1-cd0d8dba924d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:56.334613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bc6ef-193d-4521-a3c1-cd0d8dba924d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:56.334882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bc6ef-193d-4521-a3c1-cd0d8dba924d None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:56.335492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035ed36-94e6-4e1a-9d85-fd7bc36d6a91 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:56.335611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bc6ef-193d-4521-a3c1-cd0d8dba924d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:56.335846 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bc6ef-193d-4521-a3c1-cd0d8dba924d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:56.337332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bc6ef-193d-4521-a3c1-cd0d8dba924d None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:56.366560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 293/592] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:56.372974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bc6ef-193d-4521-a3c1-cd0d8dba924d None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:56.394333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c16302-0b1a-40db-871e-d654462bd5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:56.396376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c16302-0b1a-40db-871e-d654462bd5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:56.396726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c16302-0b1a-40db-871e-d654462bd5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:56.397031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c16302-0b1a-40db-871e-d654462bd5e3 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:56.397723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c16302-0b1a-40db-871e-d654462bd5e3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:56.398029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c16302-0b1a-40db-871e-d654462bd5e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:56.399465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c16302-0b1a-40db-871e-d654462bd5e3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:56.414341 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 300/593] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Oct 18 11:27:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:56.420500 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c16302-0b1a-40db-871e-d654462bd5e3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:56.451066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:56.505960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:56.558729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bc6ef-193d-4521-a3c1-cd0d8dba924d None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:56.569599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c16302-0b1a-40db-871e-d654462bd5e3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:56.616791 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811e124-e6c4-4903-bb3b-d70e8193f034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:56.617461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811e124-e6c4-4903-bb3b-d70e8193f034 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:56.617739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811e124-e6c4-4903-bb3b-d70e8193f034 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:56.618010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811e124-e6c4-4903-bb3b-d70e8193f034 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:56.633574 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811e124-e6c4-4903-bb3b-d70e8193f034 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:56.635034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811e124-e6c4-4903-bb3b-d70e8193f034 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:56.637047 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae688bb-2fc0-4289-a095-9c94ba7407c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:56.637672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae688bb-2fc0-4289-a095-9c94ba7407c1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:56.637978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae688bb-2fc0-4289-a095-9c94ba7407c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:56.638839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae688bb-2fc0-4289-a095-9c94ba7407c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:56.639560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae688bb-2fc0-4289-a095-9c94ba7407c1 None admin] RBAC: Adding query filter params (interface=public, service_id=1da8f7280129431988f27fa132d2ef9f, region_id=RegionOne) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:56.639867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae688bb-2fc0-4289-a095-9c94ba7407c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:56.642513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae688bb-2fc0-4289-a095-9c94ba7407c1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:56.659127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 301/594] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Oct 18 11:27:56 2022] GET /identity/v3/endpoints?service_id=1da8f7280129431988f27fa132d2ef9f&interface=public®ion_id=RegionOne => generated 194 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-7811e124-e6c4-4903-bb3b-d70e8193f034 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:56.661707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:56.662647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:56.663616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:56.663616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:56.663616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:56.663616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:56.663616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:56.663616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:56.663616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:56.663616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:56.663616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 18 11:27:56.663616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:56.665449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148ac33-0575-4a65-8dbd-40a93839d684 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:56.665935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148ac33-0575-4a65-8dbd-40a93839d684 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:56.666260 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148ac33-0575-4a65-8dbd-40a93839d684 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:56.666897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 294/595] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:27:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:56.671014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 302/596] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:56.671187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811e124-e6c4-4903-bb3b-d70e8193f034 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:56.672550 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148ac33-0575-4a65-8dbd-40a93839d684 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:56.765378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:56.796994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148ac33-0575-4a65-8dbd-40a93839d684 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:56.798113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811e124-e6c4-4903-bb3b-d70e8193f034 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:56.828277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:56.843116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545c65d-f5c1-41c7-886a-b1170b97aaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:56.843775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545c65d-f5c1-41c7-886a-b1170b97aaab None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:56.844050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545c65d-f5c1-41c7-886a-b1170b97aaab None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:56.844317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545c65d-f5c1-41c7-886a-b1170b97aaab None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:56.844954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545c65d-f5c1-41c7-886a-b1170b97aaab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:56.845998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fedd5-3d6e-4b5f-b049-d51bcdddbf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:56.846330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545c65d-f5c1-41c7-886a-b1170b97aaab None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:56.847264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fedd5-3d6e-4b5f-b049-d51bcdddbf6e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:56.848093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fedd5-3d6e-4b5f-b049-d51bcdddbf6e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:56.849159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fedd5-3d6e-4b5f-b049-d51bcdddbf6e None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:56.851120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fedd5-3d6e-4b5f-b049-d51bcdddbf6e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:56.851938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fedd5-3d6e-4b5f-b049-d51bcdddbf6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:56.860873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 295/597] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Oct 18 11:27:56 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:27:56.860873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a773ab-69b4-423b-b395-afb4cb846ad7 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:56.860873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a773ab-69b4-423b-b395-afb4cb846ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:56.860873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a773ab-69b4-423b-b395-afb4cb846ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:56.862913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fedd5-3d6e-4b5f-b049-d51bcdddbf6e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:56.876191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-68a773ab-69b4-423b-b395-afb4cb846ad7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:56.877465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-68a773ab-69b4-423b-b395-afb4cb846ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:56.897295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 303/598] 10.222.0.17 () {56 vars in 1135 bytes} [Tue Oct 18 11:27:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:56.902969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fedd5-3d6e-4b5f-b049-d51bcdddbf6e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:57.054838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fedd5-3d6e-4b5f-b049-d51bcdddbf6e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:57.067306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 296/599] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:27:57.119650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecead999-8140-4f68-9154-1b2f7e3459df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:57.120278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecead999-8140-4f68-9154-1b2f7e3459df None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:57.120553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecead999-8140-4f68-9154-1b2f7e3459df None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:57.120825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecead999-8140-4f68-9154-1b2f7e3459df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:57.123752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:57.203256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecead999-8140-4f68-9154-1b2f7e3459df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:57.205553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecead999-8140-4f68-9154-1b2f7e3459df None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:57.316147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 304/600] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 18 11:27:57.321397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a773ab-69b4-423b-b395-afb4cb846ad7 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:57.413818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecead999-8140-4f68-9154-1b2f7e3459df None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:57.457424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:57.526831 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a773ab-69b4-423b-b395-afb4cb846ad7 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:57.537700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecead999-8140-4f68-9154-1b2f7e3459df None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:57.567526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:57.592700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc7f50-39ab-4e1e-af3c-25b0919d238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:57.593752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc7f50-39ab-4e1e-af3c-25b0919d238f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:57.594239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc7f50-39ab-4e1e-af3c-25b0919d238f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:57.594725 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc7f50-39ab-4e1e-af3c-25b0919d238f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:57.607156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba98a78-593e-4fc2-9e9b-6a2e5c5174de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:57.608168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba98a78-593e-4fc2-9e9b-6a2e5c5174de None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:57.608676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba98a78-593e-4fc2-9e9b-6a2e5c5174de None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:57.609171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba98a78-593e-4fc2-9e9b-6a2e5c5174de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:57.615428 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc7f50-39ab-4e1e-af3c-25b0919d238f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:57.617006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc7f50-39ab-4e1e-af3c-25b0919d238f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:57.625374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba98a78-593e-4fc2-9e9b-6a2e5c5174de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-0ba98a78-593e-4fc2-9e9b-6a2e5c5174de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:27:57.633294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:57.634259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:27:57.635128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:57.637832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 305/601] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:27:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-61cc7f50-39ab-4e1e-af3c-25b0919d238f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:57.642893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:57.643867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:57.644747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:57.644747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:57.644747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:57.647126 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba98a78-593e-4fc2-9e9b-6a2e5c5174de None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:57.648788 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 297/602] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:57.649712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc7f50-39ab-4e1e-af3c-25b0919d238f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:57.758801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:57.767287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba98a78-593e-4fc2-9e9b-6a2e5c5174de None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:57.812345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7973e-34e5-4bd9-bb15-0ccd89dede72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:57.813553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7973e-34e5-4bd9-bb15-0ccd89dede72 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:57.814048 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7973e-34e5-4bd9-bb15-0ccd89dede72 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:57.814558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7973e-34e5-4bd9-bb15-0ccd89dede72 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:57.815499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7973e-34e5-4bd9-bb15-0ccd89dede72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:57.816024 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7973e-34e5-4bd9-bb15-0ccd89dede72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:57.817759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7973e-34e5-4bd9-bb15-0ccd89dede72 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:57.830227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 306/603] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:27:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:57.840509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7973e-34e5-4bd9-bb15-0ccd89dede72 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:57.944224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc7f50-39ab-4e1e-af3c-25b0919d238f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:57.989030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7973e-34e5-4bd9-bb15-0ccd89dede72 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:57.989381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd1fd3-028a-46bc-8c15-30782926dc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:57.989893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd1fd3-028a-46bc-8c15-30782926dc87 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:57.990174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd1fd3-028a-46bc-8c15-30782926dc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:57.990447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd1fd3-028a-46bc-8c15-30782926dc87 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:57.991101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd1fd3-028a-46bc-8c15-30782926dc87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:57.991407 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd1fd3-028a-46bc-8c15-30782926dc87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:57.996369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd1fd3-028a-46bc-8c15-30782926dc87 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:58.010178 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 298/604] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:58.012135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db371bf-2298-47a6-b412-f3ef40aa1fc0 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.012425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db371bf-2298-47a6-b412-f3ef40aa1fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.012888 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db371bf-2298-47a6-b412-f3ef40aa1fc0 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.016913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 299/605] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:58.017988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db371bf-2298-47a6-b412-f3ef40aa1fc0 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:58.044773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8107fad-8165-4ab0-b089-4e8caf3a7e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:58.045901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8107fad-8165-4ab0-b089-4e8caf3a7e4d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.046389 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8107fad-8165-4ab0-b089-4e8caf3a7e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.047057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8107fad-8165-4ab0-b089-4e8caf3a7e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.054419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:58.062338 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8107fad-8165-4ab0-b089-4e8caf3a7e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:58.064107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8107fad-8165-4ab0-b089-4e8caf3a7e4d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-b8107fad-8165-4ab0-b089-4e8caf3a7e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:58.081026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:58.082016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:58.082909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:58.082909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:58.082909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:58.085128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 307/606] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:58.086831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5678805-5446-4fdf-bace-581d987ce29b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.087316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5678805-5446-4fdf-bace-581d987ce29b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.087753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5678805-5446-4fdf-bace-581d987ce29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.107363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-f5678805-5446-4fdf-bace-581d987ce29b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:58.108917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f5678805-5446-4fdf-bace-581d987ce29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:58.111310 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:58.142809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db371bf-2298-47a6-b412-f3ef40aa1fc0 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:58.181346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc98e5-a787-4ce4-9897-b4b81b197d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:58.182370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc98e5-a787-4ce4-9897-b4b81b197d37 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.182897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc98e5-a787-4ce4-9897-b4b81b197d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.183301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc98e5-a787-4ce4-9897-b4b81b197d37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.191967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc98e5-a787-4ce4-9897-b4b81b197d37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:58.193659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc98e5-a787-4ce4-9897-b4b81b197d37 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-3fcc98e5-a787-4ce4-9897-b4b81b197d37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:27:58.206474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:58.207535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:58.208943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:27:58.208943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:27:58.208943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:27:58.208943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:27:58.208943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:27:58.208943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:27:58.208943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:27:58.208943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:58.208943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 300/607] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:58.208943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc98e5-a787-4ce4-9897-b4b81b197d37 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:58.210977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 308/608] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:27:58.217125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5678805-5446-4fdf-bace-581d987ce29b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:58.347200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5678805-5446-4fdf-bace-581d987ce29b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:58.349713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:58.373483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc98e5-a787-4ce4-9897-b4b81b197d37 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:58.386036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd245a1-aca3-4c89-b028-0eb7b015e631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:58.387098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd245a1-aca3-4c89-b028-0eb7b015e631 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.387586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd245a1-aca3-4c89-b028-0eb7b015e631 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.388626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd245a1-aca3-4c89-b028-0eb7b015e631 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.389560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd245a1-aca3-4c89-b028-0eb7b015e631 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:58.390088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd245a1-aca3-4c89-b028-0eb7b015e631 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:58.391739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd245a1-aca3-4c89-b028-0eb7b015e631 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:58.399239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:58.405688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 309/609] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:27:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:58.411321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd245a1-aca3-4c89-b028-0eb7b015e631 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:58.439359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2a211-800f-4fe6-a8ad-1e60ddd6b738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:58.441318 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a211-800f-4fe6-a8ad-1e60ddd6b738 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.441602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a211-800f-4fe6-a8ad-1e60ddd6b738 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.442738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a211-800f-4fe6-a8ad-1e60ddd6b738 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.443420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a211-800f-4fe6-a8ad-1e60ddd6b738 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:58.443726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a211-800f-4fe6-a8ad-1e60ddd6b738 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:58.447971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a211-800f-4fe6-a8ad-1e60ddd6b738 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:58.476960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 301/610] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:58.482604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2a211-800f-4fe6-a8ad-1e60ddd6b738 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:58.596403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd245a1-aca3-4c89-b028-0eb7b015e631 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:58.615567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2a211-800f-4fe6-a8ad-1e60ddd6b738 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:58.655639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2ce94-e595-4e30-8c11-0f6d38310a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:58.656027 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fd985-68ad-4efc-9f3d-6f7ab1e46d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:58.656255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2ce94-e595-4e30-8c11-0f6d38310a44 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.656537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2ce94-e595-4e30-8c11-0f6d38310a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.656807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2ce94-e595-4e30-8c11-0f6d38310a44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.657926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fd985-68ad-4efc-9f3d-6f7ab1e46d38 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.658570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fd985-68ad-4efc-9f3d-6f7ab1e46d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.659334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fd985-68ad-4efc-9f3d-6f7ab1e46d38 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.660609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2ce94-e595-4e30-8c11-0f6d38310a44 None admin] RBAC: Adding query filter params (role.id=8d9ff9da6ad946a6bc49336777182ff9, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=4be9de8292214872a59c1466f7513539) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:58.661185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2ce94-e595-4e30-8c11-0f6d38310a44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:58.664349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2ce94-e595-4e30-8c11-0f6d38310a44 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:58.668426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fd985-68ad-4efc-9f3d-6f7ab1e46d38 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:58.670216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fd985-68ad-4efc-9f3d-6f7ab1e46d38 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:58.671388 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 310/611] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:27:58 2022] GET /identity/v3/role_assignments?user.id=4be9de8292214872a59c1466f7513539&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=8d9ff9da6ad946a6bc49336777182ff9 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:58.672304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2ce94-e595-4e30-8c11-0f6d38310a44 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-8b6fd985-68ad-4efc-9f3d-6f7ab1e46d38 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:27:58.699659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:58.701286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:58.702246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:58.702246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:58.702246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:27:58.702246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:27:58.702246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:27:58.702246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:27:58.702246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:27:58.702246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:27:58.702246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 18 11:27:58.702246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:58.707891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 302/612] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Oct 18 11:27:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:58.712216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fd985-68ad-4efc-9f3d-6f7ab1e46d38 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:58.744681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:58.823292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:58.887110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2ce94-e595-4e30-8c11-0f6d38310a44 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:58.890621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fd985-68ad-4efc-9f3d-6f7ab1e46d38 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:58.930667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5b9cb-d71c-429b-973a-19add11f5589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:58.931314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5b9cb-d71c-429b-973a-19add11f5589 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.931589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5b9cb-d71c-429b-973a-19add11f5589 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.931695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28611b3-b028-41fb-b87a-e9a0d95c552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:58.931893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5b9cb-d71c-429b-973a-19add11f5589 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.932304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28611b3-b028-41fb-b87a-e9a0d95c552e None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.932473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5b9cb-d71c-429b-973a-19add11f5589 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:58.932579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28611b3-b028-41fb-b87a-e9a0d95c552e None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.932861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28611b3-b028-41fb-b87a-e9a0d95c552e None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.933608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28611b3-b028-41fb-b87a-e9a0d95c552e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:58.933848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5b9cb-d71c-429b-973a-19add11f5589 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:58.933956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28611b3-b028-41fb-b87a-e9a0d95c552e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:58.935391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28611b3-b028-41fb-b87a-e9a0d95c552e None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-ecc5b9cb-d71c-429b-973a-19add11f5589 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:58.944766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:58.945782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:58.946674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 311/613] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Oct 18 11:27:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:58.949010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36fe3-b4c8-4ec2-8fd8-e00c552f1173 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.949279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36fe3-b4c8-4ec2-8fd8-e00c552f1173 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.949560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36fe3-b4c8-4ec2-8fd8-e00c552f1173 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.955816 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 312/614] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:58.956892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed36fe3-b4c8-4ec2-8fd8-e00c552f1173 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:58.960171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 303/615] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Oct 18 11:27:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:58.962218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d73f4-925e-4f4a-b3fe-0fbdd4618f67 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:58.962535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d73f4-925e-4f4a-b3fe-0fbdd4618f67 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:58.962819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d73f4-925e-4f4a-b3fe-0fbdd4618f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:58.984896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e43d73f4-925e-4f4a-b3fe-0fbdd4618f67 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:58.986115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e43d73f4-925e-4f4a-b3fe-0fbdd4618f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:59.067893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:59.077313 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 304/616] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:27:59.078556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d73f4-925e-4f4a-b3fe-0fbdd4618f67 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:59.126100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed36fe3-b4c8-4ec2-8fd8-e00c552f1173 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:59.175693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:59.199045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c16db-5092-43fc-9ff0-bd290244645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:59.199675 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c16db-5092-43fc-9ff0-bd290244645f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:59.199953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c16db-5092-43fc-9ff0-bd290244645f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:59.200229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c16db-5092-43fc-9ff0-bd290244645f None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:59.201464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c16db-5092-43fc-9ff0-bd290244645f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:59.201799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c16db-5092-43fc-9ff0-bd290244645f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:59.203159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c16db-5092-43fc-9ff0-bd290244645f None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:59.219828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 313/617] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:27:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:59.225738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c16db-5092-43fc-9ff0-bd290244645f None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:59.263321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d73f4-925e-4f4a-b3fe-0fbdd4618f67 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:59.325875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85a6b2-2235-497b-b97f-789a23de6c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:59.326501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85a6b2-2235-497b-b97f-789a23de6c61 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:59.326778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85a6b2-2235-497b-b97f-789a23de6c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:59.327046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85a6b2-2235-497b-b97f-789a23de6c61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:59.407759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c16db-5092-43fc-9ff0-bd290244645f None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:59.413738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:59.423063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85a6b2-2235-497b-b97f-789a23de6c61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:59.424992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85a6b2-2235-497b-b97f-789a23de6c61 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:59.457875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbc4f5-4cf3-438f-bf86-a77aea6c6a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:59.458414 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbc4f5-4cf3-438f-bf86-a77aea6c6a88 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:59.458697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbc4f5-4cf3-438f-bf86-a77aea6c6a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:59.458963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbc4f5-4cf3-438f-bf86-a77aea6c6a88 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:59.459942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbc4f5-4cf3-438f-bf86-a77aea6c6a88 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:59.460249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbc4f5-4cf3-438f-bf86-a77aea6c6a88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:59.461921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbc4f5-4cf3-438f-bf86-a77aea6c6a88 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:59.468007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 314/618] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:27:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:59.469062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbc4f5-4cf3-438f-bf86-a77aea6c6a88 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:59.567135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:59.582614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 305/619] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:27:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 18 11:27:59.583960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcac48-2367-477c-bdb3-6a3a8bfecbdb None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:59.584254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcac48-2367-477c-bdb3-6a3a8bfecbdb None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:59.584524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcac48-2367-477c-bdb3-6a3a8bfecbdb None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:59.591056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 306/620] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:27:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:27:59.591760 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcac48-2367-477c-bdb3-6a3a8bfecbdb None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:59.632185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbc4f5-4cf3-438f-bf86-a77aea6c6a88 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:59.673507 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87732922-f20c-43a7-a000-5406e54f9c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:59.674174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732922-f20c-43a7-a000-5406e54f9c3c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:59.674462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732922-f20c-43a7-a000-5406e54f9c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:59.674734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732922-f20c-43a7-a000-5406e54f9c3c None admin] PATH_INFO: `/v3/services/network` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:59.675334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87732922-f20c-43a7-a000-5406e54f9c3c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:59.677782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87732922-f20c-43a7-a000-5406e54f9c3c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-87732922-f20c-43a7-a000-5406e54f9c3c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:59.688489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:59.689579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:27:59.690429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 315/621] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:27:59 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:59.691217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27283d31-35e1-45ef-bf2d-4d05fd978b31 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:59.691217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27283d31-35e1-45ef-bf2d-4d05fd978b31 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:59.691217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27283d31-35e1-45ef-bf2d-4d05fd978b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:59.718561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-27283d31-35e1-45ef-bf2d-4d05fd978b31 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:27:59.720481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-27283d31-35e1-45ef-bf2d-4d05fd978b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:27:59.727601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:59.736210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcac48-2367-477c-bdb3-6a3a8bfecbdb None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:59.788263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005fc26-5056-44a3-bffa-42b071263a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:59.788942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005fc26-5056-44a3-bffa-42b071263a1f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:59.789225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005fc26-5056-44a3-bffa-42b071263a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:59.789557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005fc26-5056-44a3-bffa-42b071263a1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:59.801604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005fc26-5056-44a3-bffa-42b071263a1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:59.803065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005fc26-5056-44a3-bffa-42b071263a1f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:59.820015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 316/622] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:27:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-8005fc26-5056-44a3-bffa-42b071263a1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:27:59.820659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:27:59.821603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:27:59.822866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:27:59.822866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:27:59.822866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:27:59.823037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27283d31-35e1-45ef-bf2d-4d05fd978b31 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:59.826883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 307/623] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:27:59.834348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005fc26-5056-44a3-bffa-42b071263a1f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:27:59.882702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:27:59.941726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27283d31-35e1-45ef-bf2d-4d05fd978b31 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:27:59.981228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266be90-1ce7-4f5b-b037-b51cf11af69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:27:59.981810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266be90-1ce7-4f5b-b037-b51cf11af69d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:27:59.982092 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266be90-1ce7-4f5b-b037-b51cf11af69d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:27:59.982662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266be90-1ce7-4f5b-b037-b51cf11af69d None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:27:59.983397 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266be90-1ce7-4f5b-b037-b51cf11af69d None admin] RBAC: Adding query filter params (name=network) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:27:59.983796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266be90-1ce7-4f5b-b037-b51cf11af69d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:27:59.985452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266be90-1ce7-4f5b-b037-b51cf11af69d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:27:59.991274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 317/624] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:27:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:27:59.996803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266be90-1ce7-4f5b-b037-b51cf11af69d None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:00.111921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266be90-1ce7-4f5b-b037-b51cf11af69d None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:00.128340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:00.152512 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76170e1a-0cb7-41dc-8147-4ab5c3262074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:00.153329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76170e1a-0cb7-41dc-8147-4ab5c3262074 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:00.154266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76170e1a-0cb7-41dc-8147-4ab5c3262074 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:00.154638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76170e1a-0cb7-41dc-8147-4ab5c3262074 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:00.155384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76170e1a-0cb7-41dc-8147-4ab5c3262074 None admin] RBAC: Adding query filter params (type=network) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:00.155792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76170e1a-0cb7-41dc-8147-4ab5c3262074 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:00.157327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76170e1a-0cb7-41dc-8147-4ab5c3262074 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:00.163553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 318/625] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:27:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:00.167850 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76170e1a-0cb7-41dc-8147-4ab5c3262074 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:00.185648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005fc26-5056-44a3-bffa-42b071263a1f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:00.230797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693a1b-005e-4503-9b83-965224fbfb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:00.232379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693a1b-005e-4503-9b83-965224fbfb68 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:00.233146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693a1b-005e-4503-9b83-965224fbfb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:00.233578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693a1b-005e-4503-9b83-965224fbfb68 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:00.234861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1693a1b-005e-4503-9b83-965224fbfb68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:00.235588 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1693a1b-005e-4503-9b83-965224fbfb68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:00.237257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1693a1b-005e-4503-9b83-965224fbfb68 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:00.257048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 308/626] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:00.257696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693a1b-005e-4503-9b83-965224fbfb68 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:00.281971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:00.303808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76170e1a-0cb7-41dc-8147-4ab5c3262074 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:00.339958 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee449ce-5423-4190-bc0b-cb95f6e0de6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:00.341541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee449ce-5423-4190-bc0b-cb95f6e0de6b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:00.342043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee449ce-5423-4190-bc0b-cb95f6e0de6b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:00.342508 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee449ce-5423-4190-bc0b-cb95f6e0de6b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:00.343299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee449ce-5423-4190-bc0b-cb95f6e0de6b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:00.344922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee449ce-5423-4190-bc0b-cb95f6e0de6b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-6ee449ce-5423-4190-bc0b-cb95f6e0de6b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:00.354799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:00.355844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 18 11:28:00.357695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:00.360189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 319/627] 10.222.0.17 () {56 vars in 1129 bytes} [Tue Oct 18 11:28:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:00.361367 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee449ce-5423-4190-bc0b-cb95f6e0de6b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:00.394733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693a1b-005e-4503-9b83-965224fbfb68 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:00.409491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:00.445164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c2532-d638-4bd1-a7f7-7a5254bfc18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:00.445856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c2532-d638-4bd1-a7f7-7a5254bfc18a None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:00.446055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c2532-d638-4bd1-a7f7-7a5254bfc18a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:00.446323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c2532-d638-4bd1-a7f7-7a5254bfc18a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:00.446996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c2532-d638-4bd1-a7f7-7a5254bfc18a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:00.448456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c2532-d638-4bd1-a7f7-7a5254bfc18a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:00.478696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee449ce-5423-4190-bc0b-cb95f6e0de6b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:00.483519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 309/628] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:28:00 2022] POST /identity/v3/endpoints => generated 337 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:00.484629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c2532-d638-4bd1-a7f7-7a5254bfc18a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:00.540444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1e43b-538a-4659-b88c-eb1c452e812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:00.541690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1e43b-538a-4659-b88c-eb1c452e812b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:00.542336 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1e43b-538a-4659-b88c-eb1c452e812b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:00.543057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1e43b-538a-4659-b88c-eb1c452e812b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:00.553736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1e43b-538a-4659-b88c-eb1c452e812b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:00.556723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1e43b-538a-4659-b88c-eb1c452e812b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-ecb1e43b-538a-4659-b88c-eb1c452e812b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:28:00.570381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:00.571458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:00.572351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:28:00.572351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:28:00.572351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:28:00.572351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:28:00.572351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:28:00.572351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:28:00.572351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:00.572351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:00.573796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:00.576157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 320/629] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:28:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:00.582196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1e43b-538a-4659-b88c-eb1c452e812b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:00.666006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c2532-d638-4bd1-a7f7-7a5254bfc18a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:00.740433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f490e27-a212-4b0c-ba82-4c5673fc08aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:00.741330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f490e27-a212-4b0c-ba82-4c5673fc08aa None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:00.741775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f490e27-a212-4b0c-ba82-4c5673fc08aa None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:00.742217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f490e27-a212-4b0c-ba82-4c5673fc08aa None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:00.743020 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f490e27-a212-4b0c-ba82-4c5673fc08aa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:00.743754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f490e27-a212-4b0c-ba82-4c5673fc08aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:00.746618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f490e27-a212-4b0c-ba82-4c5673fc08aa None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:00.758172 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:00.764965 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 310/630] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Oct 18 11:28:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:00.769103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f490e27-a212-4b0c-ba82-4c5673fc08aa None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:00.774492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1e43b-538a-4659-b88c-eb1c452e812b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:00.834960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a555f8-395a-464f-b659-17d16141419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:00.836073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a555f8-395a-464f-b659-17d16141419c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:00.836521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a555f8-395a-464f-b659-17d16141419c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:00.837113 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a555f8-395a-464f-b659-17d16141419c None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:00.838088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a555f8-395a-464f-b659-17d16141419c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:00.838536 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a555f8-395a-464f-b659-17d16141419c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:00.840617 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a555f8-395a-464f-b659-17d16141419c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:00.857377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 321/631] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:28:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:00.866680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a555f8-395a-464f-b659-17d16141419c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:00.913326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f490e27-a212-4b0c-ba82-4c5673fc08aa None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:00.953569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:00.964954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f30cd-0b13-4e7d-b00c-25a569e18cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:00.965623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f30cd-0b13-4e7d-b00c-25a569e18cbd None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:00.966438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f30cd-0b13-4e7d-b00c-25a569e18cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:00.966850 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f30cd-0b13-4e7d-b00c-25a569e18cbd None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:00.967813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f30cd-0b13-4e7d-b00c-25a569e18cbd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:00.968123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f30cd-0b13-4e7d-b00c-25a569e18cbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:00.970227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f30cd-0b13-4e7d-b00c-25a569e18cbd None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:00.982781 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 311/632] 10.222.0.17 () {56 vars in 1124 bytes} [Tue Oct 18 11:28:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:00.986007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f30cd-0b13-4e7d-b00c-25a569e18cbd None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:01.007430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a555f8-395a-464f-b659-17d16141419c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:01.057545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d58f4a-f91f-44a1-b4ea-ae4f60d98fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:01.058166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d58f4a-f91f-44a1-b4ea-ae4f60d98fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:01.058444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d58f4a-f91f-44a1-b4ea-ae4f60d98fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:01.058715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d58f4a-f91f-44a1-b4ea-ae4f60d98fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:01.059451 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d58f4a-f91f-44a1-b4ea-ae4f60d98fb1 None admin] RBAC: Adding query filter params (role.id=f94c9d39cd6244b4b0a727e200ff2e94, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=74136098746a4c10b1a74051b75bbd2b) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:01.059797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d58f4a-f91f-44a1-b4ea-ae4f60d98fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:01.061274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d58f4a-f91f-44a1-b4ea-ae4f60d98fb1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:01.070441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 322/633] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:28:00 2022] GET /identity/v3/role_assignments?user.id=74136098746a4c10b1a74051b75bbd2b&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=f94c9d39cd6244b4b0a727e200ff2e94 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:01.121608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f889ce2-816c-4ba8-99c1-b0625c34cf21 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:01.121804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f30cd-0b13-4e7d-b00c-25a569e18cbd None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:01.121887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f889ce2-816c-4ba8-99c1-b0625c34cf21 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:01.122308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f889ce2-816c-4ba8-99c1-b0625c34cf21 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:01.124015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 323/634] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:01.129980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129a16f-4452-47bf-9883-6799bee51c10 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:01.130298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129a16f-4452-47bf-9883-6799bee51c10 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:01.130533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129a16f-4452-47bf-9883-6799bee51c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:01.151214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7129a16f-4452-47bf-9883-6799bee51c10 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:01.152804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7129a16f-4452-47bf-9883-6799bee51c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:01.171491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:01.175086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979511b-a613-4e80-a5f0-0d5dc286e9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:01.176350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979511b-a613-4e80-a5f0-0d5dc286e9d6 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:01.178658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979511b-a613-4e80-a5f0-0d5dc286e9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:01.179261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979511b-a613-4e80-a5f0-0d5dc286e9d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:01.180216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979511b-a613-4e80-a5f0-0d5dc286e9d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:01.181699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979511b-a613-4e80-a5f0-0d5dc286e9d6 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:01.211643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 312/635] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:28:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:28:01.260381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 324/636] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:28:01.528209 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979511b-a613-4e80-a5f0-0d5dc286e9d6 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:01.586661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:01.641965 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979511b-a613-4e80-a5f0-0d5dc286e9d6 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:01.678404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa2124-ce38-4e8c-9c72-26ca4672bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:01.679052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa2124-ce38-4e8c-9c72-26ca4672bb61 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:01.679330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa2124-ce38-4e8c-9c72-26ca4672bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:01.679603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa2124-ce38-4e8c-9c72-26ca4672bb61 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:01.680198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa2124-ce38-4e8c-9c72-26ca4672bb61 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:01.681632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa2124-ce38-4e8c-9c72-26ca4672bb61 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-ebfa2124-ce38-4e8c-9c72-26ca4672bb61 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:01.691844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:01.692907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:01.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 313/637] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:28:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:01.698973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129a16f-4452-47bf-9883-6799bee51c10 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:01.807605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129a16f-4452-47bf-9883-6799bee51c10 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:01.847847 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06bf02-6528-4a7d-9648-ed260bab8447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:01.848497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06bf02-6528-4a7d-9648-ed260bab8447 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:01.848774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06bf02-6528-4a7d-9648-ed260bab8447 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:01.849085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06bf02-6528-4a7d-9648-ed260bab8447 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:01.849740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06bf02-6528-4a7d-9648-ed260bab8447 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:01.850040 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06bf02-6528-4a7d-9648-ed260bab8447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:01.851465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06bf02-6528-4a7d-9648-ed260bab8447 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:01.857459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 325/638] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:28:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:01.862899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa2124-ce38-4e8c-9c72-26ca4672bb61 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:01.970731 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69a875-d38d-4367-ad0d-8c38653ae2d6 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:01.971059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69a875-d38d-4367-ad0d-8c38653ae2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:01.971913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69a875-d38d-4367-ad0d-8c38653ae2d6 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:01.976936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 326/639] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:01.978389 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b9023-2ad3-4b17-9df5-b61cf858dad1 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:01.978685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b9023-2ad3-4b17-9df5-b61cf858dad1 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:01.978959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b9023-2ad3-4b17-9df5-b61cf858dad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:01.981715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa2124-ce38-4e8c-9c72-26ca4672bb61 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:02.000047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:02.001854 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d98b9023-2ad3-4b17-9df5-b61cf858dad1 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:02.003384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d98b9023-2ad3-4b17-9df5-b61cf858dad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:02.007160 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:02.033995 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc27498-a224-4168-97db-362dacc070e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:02.035024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc27498-a224-4168-97db-362dacc070e9 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:02.035528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc27498-a224-4168-97db-362dacc070e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:02.036002 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc27498-a224-4168-97db-362dacc070e9 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:02.036893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc27498-a224-4168-97db-362dacc070e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:02.037402 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc27498-a224-4168-97db-362dacc070e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:02.038983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc27498-a224-4168-97db-362dacc070e9 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:02.049352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 314/640] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:28:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:02.095406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 327/641] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:02.369365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc27498-a224-4168-97db-362dacc070e9 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:02.483211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc27498-a224-4168-97db-362dacc070e9 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:02.520582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186f7d7-dc68-4559-a6cf-30c4c31f346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:02.521230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186f7d7-dc68-4559-a6cf-30c4c31f346b None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:02.521510 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186f7d7-dc68-4559-a6cf-30c4c31f346b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:02.521782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186f7d7-dc68-4559-a6cf-30c4c31f346b None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:02.522444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186f7d7-dc68-4559-a6cf-30c4c31f346b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:02.523806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186f7d7-dc68-4559-a6cf-30c4c31f346b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:02.541826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 315/642] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:28:02 2022] POST /identity/v3/services => generated 269 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:03.372294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d6e88-df8a-439b-a90f-5dfa680ff2a9 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:03.372809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d6e88-df8a-439b-a90f-5dfa680ff2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:03.372946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d6e88-df8a-439b-a90f-5dfa680ff2a9 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:03.374445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 328/643] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:03.380243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6456ca2-07f2-4a50-9a9d-4503b9637b01 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:03.380547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6456ca2-07f2-4a50-9a9d-4503b9637b01 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:03.380806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6456ca2-07f2-4a50-9a9d-4503b9637b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:03.401471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c6456ca2-07f2-4a50-9a9d-4503b9637b01 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:03.402746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c6456ca2-07f2-4a50-9a9d-4503b9637b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:03.443240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:03.489543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 316/644] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:03 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:28:03.534127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467beb07-5d79-493a-90f9-e6df4e2ae9ae None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:03.534455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467beb07-5d79-493a-90f9-e6df4e2ae9ae None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:03.534665 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467beb07-5d79-493a-90f9-e6df4e2ae9ae None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:03.536189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 329/645] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:03.544991 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1de78-2dfa-4787-9e3a-c5834186dcc0 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:03.546280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1de78-2dfa-4787-9e3a-c5834186dcc0 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:03.546972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1de78-2dfa-4787-9e3a-c5834186dcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:03.574597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1fa1de78-2dfa-4787-9e3a-c5834186dcc0 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:03.577388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1fa1de78-2dfa-4787-9e3a-c5834186dcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:03.656223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 317/646] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:03 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:03.780271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467beb07-5d79-493a-90f9-e6df4e2ae9ae None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:03.894449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467beb07-5d79-493a-90f9-e6df4e2ae9ae None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:03.929119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1de78-2dfa-4787-9e3a-c5834186dcc0 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:03.931855 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa0d96-d59a-45d7-9573-b063c84c508b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:03.932504 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa0d96-d59a-45d7-9573-b063c84c508b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:03.932783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa0d96-d59a-45d7-9573-b063c84c508b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:03.933073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa0d96-d59a-45d7-9573-b063c84c508b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:03.946297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:03.958829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa0d96-d59a-45d7-9573-b063c84c508b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-0eaa0d96-d59a-45d7-9573-b063c84c508b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:28:03.962445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:03.963393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:03.964248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 330/647] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Oct 18 11:28:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:03.970616 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa0d96-d59a-45d7-9573-b063c84c508b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:04.040946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:04.049217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1de78-2dfa-4787-9e3a-c5834186dcc0 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:04.093009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34f34a-adc3-4ef1-ae5b-50f6743aaf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:04.094000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34f34a-adc3-4ef1-ae5b-50f6743aaf28 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.094488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34f34a-adc3-4ef1-ae5b-50f6743aaf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.094858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34f34a-adc3-4ef1-ae5b-50f6743aaf28 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.095637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34f34a-adc3-4ef1-ae5b-50f6743aaf28 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:04.097430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34f34a-adc3-4ef1-ae5b-50f6743aaf28 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:04.107679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa0d96-d59a-45d7-9573-b063c84c508b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-aa34f34a-adc3-4ef1-ae5b-50f6743aaf28 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:04.109608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:04.111021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 18 11:28:04.112447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:04.116893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 318/648] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:28:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:04.116893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34f34a-adc3-4ef1-ae5b-50f6743aaf28 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:04.165359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fc4dd-8aed-4efe-8c53-dd4fc5cfb046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:04.166402 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fc4dd-8aed-4efe-8c53-dd4fc5cfb046 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.166919 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fc4dd-8aed-4efe-8c53-dd4fc5cfb046 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.167429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fc4dd-8aed-4efe-8c53-dd4fc5cfb046 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.169529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fc4dd-8aed-4efe-8c53-dd4fc5cfb046 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:04.170044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fc4dd-8aed-4efe-8c53-dd4fc5cfb046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:04.172440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fc4dd-8aed-4efe-8c53-dd4fc5cfb046 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:04.189551 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 331/649] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Oct 18 11:28:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:04.191061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fc4dd-8aed-4efe-8c53-dd4fc5cfb046 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:04.238195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34f34a-adc3-4ef1-ae5b-50f6743aaf28 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:04.281249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:04.282768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86d74a-17cd-4784-86c4-0bd3218ac2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:04.283329 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86d74a-17cd-4784-86c4-0bd3218ac2ee None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.283606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86d74a-17cd-4784-86c4-0bd3218ac2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.283878 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86d74a-17cd-4784-86c4-0bd3218ac2ee None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.284512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86d74a-17cd-4784-86c4-0bd3218ac2ee None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:04.284890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86d74a-17cd-4784-86c4-0bd3218ac2ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:04.286310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86d74a-17cd-4784-86c4-0bd3218ac2ee None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:04.297360 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 319/650] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:28:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:04.300471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86d74a-17cd-4784-86c4-0bd3218ac2ee None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:04.328262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fc4dd-8aed-4efe-8c53-dd4fc5cfb046 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:04.329264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:04.373232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582e50f-b235-463b-9196-58afcf1d07af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:04.374628 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582e50f-b235-463b-9196-58afcf1d07af None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.374980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582e50f-b235-463b-9196-58afcf1d07af None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.375266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582e50f-b235-463b-9196-58afcf1d07af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.386728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582e50f-b235-463b-9196-58afcf1d07af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:04.388811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582e50f-b235-463b-9196-58afcf1d07af None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-d582e50f-b235-463b-9196-58afcf1d07af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:28:04.404639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:04.405809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:28:04.406701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:28:04.406701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:04.406701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:04.406701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 332/651] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:04.407300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9b35c-c5c3-4415-8a71-6bb85b97b8b2 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.407300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9b35c-c5c3-4415-8a71-6bb85b97b8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.407678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9b35c-c5c3-4415-8a71-6bb85b97b8b2 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.408946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 333/652] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:04.413728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9b35c-c5c3-4415-8a71-6bb85b97b8b2 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:04.418467 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86d74a-17cd-4784-86c4-0bd3218ac2ee None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:04.461794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27962d2a-b43d-4b61-bf1e-30d8fa30797a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:04.463250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27962d2a-b43d-4b61-bf1e-30d8fa30797a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.463250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27962d2a-b43d-4b61-bf1e-30d8fa30797a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.463250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27962d2a-b43d-4b61-bf1e-30d8fa30797a None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.463694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27962d2a-b43d-4b61-bf1e-30d8fa30797a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:04.464021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27962d2a-b43d-4b61-bf1e-30d8fa30797a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:04.465413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27962d2a-b43d-4b61-bf1e-30d8fa30797a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:04.477280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 320/653] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:28:04 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:04.479172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a8588-9135-4bbb-8a68-8e756897d1b2 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.479454 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a8588-9135-4bbb-8a68-8e756897d1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.479733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a8588-9135-4bbb-8a68-8e756897d1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.505168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-be0a8588-9135-4bbb-8a68-8e756897d1b2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:04.506582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-be0a8588-9135-4bbb-8a68-8e756897d1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:04.544481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9b35c-c5c3-4415-8a71-6bb85b97b8b2 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:04.566458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:04.604910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589c304-d5a8-4732-bf08-7ab01148de00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:04.604910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589c304-d5a8-4732-bf08-7ab01148de00 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.604910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589c304-d5a8-4732-bf08-7ab01148de00 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.604910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589c304-d5a8-4732-bf08-7ab01148de00 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.605345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589c304-d5a8-4732-bf08-7ab01148de00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:04.605345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589c304-d5a8-4732-bf08-7ab01148de00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:04.605345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589c304-d5a8-4732-bf08-7ab01148de00 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:04.606234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 321/654] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:04 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:04.617093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 334/655] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:04.622377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a8588-9135-4bbb-8a68-8e756897d1b2 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:04.657308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:04.741207 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a8588-9135-4bbb-8a68-8e756897d1b2 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:04.782311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef6410-496a-42f0-9488-717a920f1048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:04.783321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef6410-496a-42f0-9488-717a920f1048 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.783826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef6410-496a-42f0-9488-717a920f1048 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.784309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef6410-496a-42f0-9488-717a920f1048 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.793036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef6410-496a-42f0-9488-717a920f1048 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:04.795089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef6410-496a-42f0-9488-717a920f1048 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-7bef6410-496a-42f0-9488-717a920f1048 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:28:04.808262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:04.809331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:04.810886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:04.810886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:04.810886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:28:04.810886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:28:04.810886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:28:04.810886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:28:04.810886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:28:04.810886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:28:04.810886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 18 11:28:04.810886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:04.813521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 322/656] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Oct 18 11:28:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:04.819484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589c304-d5a8-4732-bf08-7ab01148de00 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:04.853483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93613937-7ecf-49db-904b-4c1872ce4f50 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.853813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93613937-7ecf-49db-904b-4c1872ce4f50 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.854032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93613937-7ecf-49db-904b-4c1872ce4f50 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.855641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 323/657] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:04.860983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f785-ac1b-43d7-aac9-f97a43e8f65e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.861300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f785-ac1b-43d7-aac9-f97a43e8f65e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.861520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f785-ac1b-43d7-aac9-f97a43e8f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.880703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1723f785-ac1b-43d7-aac9-f97a43e8f65e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:04.881940 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1723f785-ac1b-43d7-aac9-f97a43e8f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:04.932318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589c304-d5a8-4732-bf08-7ab01148de00 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:04.956327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 324/658] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:04 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:28:04.957273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723f785-ac1b-43d7-aac9-f97a43e8f65e None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:04.973734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c9858-c2ea-4264-b10e-f6d45c8dfd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:04.974047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:04.974722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c9858-c2ea-4264-b10e-f6d45c8dfd4b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:04.974835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c9858-c2ea-4264-b10e-f6d45c8dfd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:04.975161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c9858-c2ea-4264-b10e-f6d45c8dfd4b None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:04.975996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c9858-c2ea-4264-b10e-f6d45c8dfd4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:04.976366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c9858-c2ea-4264-b10e-f6d45c8dfd4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:04.978018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c9858-c2ea-4264-b10e-f6d45c8dfd4b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:04.999838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 335/659] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Oct 18 11:28:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:05.005655 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c9858-c2ea-4264-b10e-f6d45c8dfd4b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:05.040187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:05.101947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723f785-ac1b-43d7-aac9-f97a43e8f65e None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:05.127632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c9858-c2ea-4264-b10e-f6d45c8dfd4b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:05.148330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e79d63-3db1-4100-984b-973464fe7c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:05.149585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e79d63-3db1-4100-984b-973464fe7c44 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:05.149920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e79d63-3db1-4100-984b-973464fe7c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:05.150308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e79d63-3db1-4100-984b-973464fe7c44 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:05.151060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e79d63-3db1-4100-984b-973464fe7c44 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:05.152547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e79d63-3db1-4100-984b-973464fe7c44 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:05.172911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f46c4f-fd3e-40aa-90d3-0c1ad619eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:05.173890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f46c4f-fd3e-40aa-90d3-0c1ad619eea5 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:05.174357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f46c4f-fd3e-40aa-90d3-0c1ad619eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:05.174962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f46c4f-fd3e-40aa-90d3-0c1ad619eea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:05.177074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 325/660] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:28:04 2022] POST /identity/v3/services => generated 241 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:05.194890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f46c4f-fd3e-40aa-90d3-0c1ad619eea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:05.197197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f46c4f-fd3e-40aa-90d3-0c1ad619eea5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-d0f46c4f-fd3e-40aa-90d3-0c1ad619eea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:28:05.221569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:05.222581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:28:05.223462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:28:05.223462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:05.223462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:05.226176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 336/661] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:05.232504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e79d63-3db1-4100-984b-973464fe7c44 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:05.281993 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f46c4f-fd3e-40aa-90d3-0c1ad619eea5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:05.327260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:05.390891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e79d63-3db1-4100-984b-973464fe7c44 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:05.415386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:05.438000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f46c4f-fd3e-40aa-90d3-0c1ad619eea5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:05.440362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f097e13-9387-4751-9ee0-5a98e3db071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:05.441476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f097e13-9387-4751-9ee0-5a98e3db071b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:05.441476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f097e13-9387-4751-9ee0-5a98e3db071b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:05.441688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f097e13-9387-4751-9ee0-5a98e3db071b None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:05.443263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f097e13-9387-4751-9ee0-5a98e3db071b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:05.443263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f097e13-9387-4751-9ee0-5a98e3db071b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:05.444545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f097e13-9387-4751-9ee0-5a98e3db071b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:05.458251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 326/662] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:05.463875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f097e13-9387-4751-9ee0-5a98e3db071b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:05.482012 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e2a3d-07bd-48cb-bb1b-278622acfa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:05.483144 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e2a3d-07bd-48cb-bb1b-278622acfa80 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:05.483776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e2a3d-07bd-48cb-bb1b-278622acfa80 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:05.484058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e2a3d-07bd-48cb-bb1b-278622acfa80 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:05.484655 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e2a3d-07bd-48cb-bb1b-278622acfa80 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:05.486793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e2a3d-07bd-48cb-bb1b-278622acfa80 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-b86e2a3d-07bd-48cb-bb1b-278622acfa80 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:05.500261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:05.501267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:05.502145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 337/663] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Oct 18 11:28:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:05.509100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e2a3d-07bd-48cb-bb1b-278622acfa80 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:05.598933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f097e13-9387-4751-9ee0-5a98e3db071b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:05.628466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:05.643844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6cbff-aa96-4fc7-96d8-ffb296777060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:05.644493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6cbff-aa96-4fc7-96d8-ffb296777060 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:05.644777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6cbff-aa96-4fc7-96d8-ffb296777060 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:05.645083 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6cbff-aa96-4fc7-96d8-ffb296777060 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:05.655567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6cbff-aa96-4fc7-96d8-ffb296777060 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:05.656703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e2a3d-07bd-48cb-bb1b-278622acfa80 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:05.657083 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6cbff-aa96-4fc7-96d8-ffb296777060 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-1ef6cbff-aa96-4fc7-96d8-ffb296777060 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:28:05.670115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:05.671084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:05.671956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:28:05.671956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:28:05.671956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:28:05.671956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:28:05.671956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:28:05.671956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:28:05.671956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:05.671956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:05.676092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 327/664] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:28:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:05.677767 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6cbff-aa96-4fc7-96d8-ffb296777060 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:05.705122 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970e198-0a9b-4787-b4ab-3196418d2fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:05.705738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970e198-0a9b-4787-b4ab-3196418d2fae None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:05.706013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970e198-0a9b-4787-b4ab-3196418d2fae None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:05.706343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970e198-0a9b-4787-b4ab-3196418d2fae None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:05.707218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970e198-0a9b-4787-b4ab-3196418d2fae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:05.707347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970e198-0a9b-4787-b4ab-3196418d2fae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:05.708618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970e198-0a9b-4787-b4ab-3196418d2fae None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:05.719441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 338/665] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:28:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:05.720729 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970e198-0a9b-4787-b4ab-3196418d2fae None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:05.741347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:05.810648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6cbff-aa96-4fc7-96d8-ffb296777060 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:05.844136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970e198-0a9b-4787-b4ab-3196418d2fae None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:05.851253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6adf6-375d-4ab4-9edf-fe020f817b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:05.852651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6adf6-375d-4ab4-9edf-fe020f817b13 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:05.853333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6adf6-375d-4ab4-9edf-fe020f817b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:05.854169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6adf6-375d-4ab4-9edf-fe020f817b13 None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:05.855402 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6adf6-375d-4ab4-9edf-fe020f817b13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:05.855920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6adf6-375d-4ab4-9edf-fe020f817b13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:05.858346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6adf6-375d-4ab4-9edf-fe020f817b13 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:05.873579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 328/666] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:28:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:05.879566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6adf6-375d-4ab4-9edf-fe020f817b13 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:05.886961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6823bb-dd3e-48ba-8158-676249b414a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:05.887607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6823bb-dd3e-48ba-8158-676249b414a9 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:05.887885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6823bb-dd3e-48ba-8158-676249b414a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:05.888434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6823bb-dd3e-48ba-8158-676249b414a9 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:05.889152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6823bb-dd3e-48ba-8158-676249b414a9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:05.889450 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6823bb-dd3e-48ba-8158-676249b414a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:05.891080 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6823bb-dd3e-48ba-8158-676249b414a9 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:05.901188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 339/667] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:28:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:05.902075 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6823bb-dd3e-48ba-8158-676249b414a9 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:05.953263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:06.008656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:06.009073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6adf6-375d-4ab4-9edf-fe020f817b13 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:06.042099 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6823bb-dd3e-48ba-8158-676249b414a9 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:06.053341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37dca2-8900-4fdd-a7da-00a7cbcd01a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:06.054306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37dca2-8900-4fdd-a7da-00a7cbcd01a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:06.054419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37dca2-8900-4fdd-a7da-00a7cbcd01a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:06.054662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37dca2-8900-4fdd-a7da-00a7cbcd01a7 None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/74136098746a4c10b1a74051b75bbd2b/roles/67d4f6145bce41c6a2168196bb6d8a74` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:06.057199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37dca2-8900-4fdd-a7da-00a7cbcd01a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=74136098746a4c10b1a74051b75bbd2b, role_id=67d4f6145bce41c6a2168196bb6d8a74)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:06.058487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37dca2-8900-4fdd-a7da-00a7cbcd01a7 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:06.076947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 329/668] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:28:05 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/74136098746a4c10b1a74051b75bbd2b/roles/67d4f6145bce41c6a2168196bb6d8a74 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:28:06.095155 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4b4b2-ff0d-4ea2-a46f-22f73440ccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:06.096295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b4b2-ff0d-4ea2-a46f-22f73440ccde None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:06.096792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b4b2-ff0d-4ea2-a46f-22f73440ccde None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:06.097373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b4b2-ff0d-4ea2-a46f-22f73440ccde None admin] PATH_INFO: `/v3/endpoints` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:06.098448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4b4b2-ff0d-4ea2-a46f-22f73440ccde None admin] RBAC: Adding query filter params (interface=public, service_id=edba6b799b204da6b8b35469cc4cb6fb, region_id=RegionOne) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:06.099054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4b4b2-ff0d-4ea2-a46f-22f73440ccde None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:06.100967 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4b4b2-ff0d-4ea2-a46f-22f73440ccde None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:06.109077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 340/669] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Oct 18 11:28:05 2022] GET /identity/v3/endpoints?service_id=edba6b799b204da6b8b35469cc4cb6fb&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:06.115621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37dca2-8900-4fdd-a7da-00a7cbcd01a7 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:06.247925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37dca2-8900-4fdd-a7da-00a7cbcd01a7 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:06.251411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:06.289851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524aa52-5070-4090-b568-a9d3f93c5acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:06.290712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524aa52-5070-4090-b568-a9d3f93c5acf None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:06.291093 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524aa52-5070-4090-b568-a9d3f93c5acf None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:06.291461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524aa52-5070-4090-b568-a9d3f93c5acf None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:06.292191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524aa52-5070-4090-b568-a9d3f93c5acf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:06.294406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524aa52-5070-4090-b568-a9d3f93c5acf None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:06.302838 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 330/670] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Oct 18 11:28:06 2022] GET /identity/v3/services => generated 1258 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:28:06.742296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d84fd-3653-4084-a8db-623b366fe62b None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:06.742838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d84fd-3653-4084-a8db-623b366fe62b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:06.742838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d84fd-3653-4084-a8db-623b366fe62b None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:06.744382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 341/671] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:06.749780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d3777-4835-498f-bb93-40489cf7b2d1 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:06.750166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d3777-4835-498f-bb93-40489cf7b2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:06.750344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d3777-4835-498f-bb93-40489cf7b2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:06.768666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e51d3777-4835-498f-bb93-40489cf7b2d1 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:06.769979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e51d3777-4835-498f-bb93-40489cf7b2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:06.839594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 331/672] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:06 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:07.108888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d84fd-3653-4084-a8db-623b366fe62b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:07.226691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d84fd-3653-4084-a8db-623b366fe62b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:07.267316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca591493-a150-4de6-ba35-03a129994cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:07.268288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca591493-a150-4de6-ba35-03a129994cf9 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:07.268760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca591493-a150-4de6-ba35-03a129994cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:07.269315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca591493-a150-4de6-ba35-03a129994cf9 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:07.270200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca591493-a150-4de6-ba35-03a129994cf9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:07.271779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca591493-a150-4de6-ba35-03a129994cf9 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:07.290411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:07.297296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 342/673] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:28:07 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:07.511335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea8a0d-587d-49fa-ad63-f912ce6e0be1 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:07.511710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea8a0d-587d-49fa-ad63-f912ce6e0be1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:07.511908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea8a0d-587d-49fa-ad63-f912ce6e0be1 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:07.513571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 332/674] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:07.519028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525e33-2699-48ff-8023-7b684bb0a62d None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:07.519315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525e33-2699-48ff-8023-7b684bb0a62d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:07.519590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525e33-2699-48ff-8023-7b684bb0a62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:07.538043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b7525e33-2699-48ff-8023-7b684bb0a62d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:07.539248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b7525e33-2699-48ff-8023-7b684bb0a62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:07.626407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 343/675] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:07.909062 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea8a0d-587d-49fa-ad63-f912ce6e0be1 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:08.020263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:08.026333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea8a0d-587d-49fa-ad63-f912ce6e0be1 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:08.062913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b2b46-7360-4aad-b021-74a4a5a4f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:08.063904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b2b46-7360-4aad-b021-74a4a5a4f986 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:08.064383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b2b46-7360-4aad-b021-74a4a5a4f986 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:08.064890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b2b46-7360-4aad-b021-74a4a5a4f986 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:08.065670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b2b46-7360-4aad-b021-74a4a5a4f986 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:08.067250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b2b46-7360-4aad-b021-74a4a5a4f986 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-046b2b46-7360-4aad-b021-74a4a5a4f986 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:08.076814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:08.077843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:08.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:08.081114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 333/676] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:28:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:08.092789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7525e33-2699-48ff-8023-7b684bb0a62d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:08.203794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7525e33-2699-48ff-8023-7b684bb0a62d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:08.230883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:08.241107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcda5d7-00b3-426b-adb3-bd62b464ff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:08.241952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcda5d7-00b3-426b-adb3-bd62b464ff17 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:08.242231 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcda5d7-00b3-426b-adb3-bd62b464ff17 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:08.242501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcda5d7-00b3-426b-adb3-bd62b464ff17 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:08.243135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcda5d7-00b3-426b-adb3-bd62b464ff17 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:08.243434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcda5d7-00b3-426b-adb3-bd62b464ff17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:08.244814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcda5d7-00b3-426b-adb3-bd62b464ff17 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:08.250735 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 344/677] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:28:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:08.256672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b2b46-7360-4aad-b021-74a4a5a4f986 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:08.365539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b2b46-7360-4aad-b021-74a4a5a4f986 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:08.408410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6cc87-804c-4dd1-aae3-d046a828c54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:08.409045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6cc87-804c-4dd1-aae3-d046a828c54b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:08.409324 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6cc87-804c-4dd1-aae3-d046a828c54b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:08.409592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6cc87-804c-4dd1-aae3-d046a828c54b None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:08.410218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6cc87-804c-4dd1-aae3-d046a828c54b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:08.410514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6cc87-804c-4dd1-aae3-d046a828c54b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:08.412032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6cc87-804c-4dd1-aae3-d046a828c54b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:08.417194 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c0bd-edb4-4726-aaa8-cca3988cf56e None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:08.417482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c0bd-edb4-4726-aaa8-cca3988cf56e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:08.417761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c0bd-edb4-4726-aaa8-cca3988cf56e None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:08.417930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 334/678] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:28:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:08.419203 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 345/679] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:08.424255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6cc87-804c-4dd1-aae3-d046a828c54b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:08.424978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552980a-f3a5-469f-bcad-5fe73bedd67a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:08.425529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552980a-f3a5-469f-bcad-5fe73bedd67a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:08.426011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552980a-f3a5-469f-bcad-5fe73bedd67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:08.446871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-1552980a-f3a5-469f-bcad-5fe73bedd67a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:08.447993 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-1552980a-f3a5-469f-bcad-5fe73bedd67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:08.461402 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:08.536858 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 346/680] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:08.564074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6cc87-804c-4dd1-aae3-d046a828c54b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:08.602703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedb469-b080-4bcd-a7c5-3402e91564ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:08.603399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedb469-b080-4bcd-a7c5-3402e91564ff None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:08.603685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedb469-b080-4bcd-a7c5-3402e91564ff None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:08.603965 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedb469-b080-4bcd-a7c5-3402e91564ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:08.604680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedb469-b080-4bcd-a7c5-3402e91564ff None admin] RBAC: Adding query filter params (interface=public, service_id=25684b799fa643609d07e8686d115576, region_id=RegionOne) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:08.605022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedb469-b080-4bcd-a7c5-3402e91564ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:08.606452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedb469-b080-4bcd-a7c5-3402e91564ff None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:08.614581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 335/681] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Oct 18 11:28:08 2022] GET /identity/v3/endpoints?service_id=25684b799fa643609d07e8686d115576&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:08.621234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552980a-f3a5-469f-bcad-5fe73bedd67a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:08.630979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4c51a-1f35-4179-b90e-9515ccf237ad None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:08.631722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4c51a-1f35-4179-b90e-9515ccf237ad None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:08.632202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4c51a-1f35-4179-b90e-9515ccf237ad None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:08.634178 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 336/682] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:08.640561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347736d2-1f94-4714-b105-b67230ea46e7 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:08.641180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347736d2-1f94-4714-b105-b67230ea46e7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:08.641657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347736d2-1f94-4714-b105-b67230ea46e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:08.662186 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-347736d2-1f94-4714-b105-b67230ea46e7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:08.663841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-347736d2-1f94-4714-b105-b67230ea46e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:08.749351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 337/683] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:08.763723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552980a-f3a5-469f-bcad-5fe73bedd67a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:08.787959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:08.803906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb1b9b-616f-4778-8bb9-7bb65adea95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:08.804537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb1b9b-616f-4778-8bb9-7bb65adea95b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:08.804816 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb1b9b-616f-4778-8bb9-7bb65adea95b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:08.805119 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb1b9b-616f-4778-8bb9-7bb65adea95b None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:08.805739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb1b9b-616f-4778-8bb9-7bb65adea95b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:08.807141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb1b9b-616f-4778-8bb9-7bb65adea95b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:08.815078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 347/684] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Oct 18 11:28:08 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:28:08.822021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347736d2-1f94-4714-b105-b67230ea46e7 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:08.850202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:08.942660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347736d2-1f94-4714-b105-b67230ea46e7 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:08.981744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7548d-6a37-4d52-a3d2-7d41f042604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:08.982805 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7548d-6a37-4d52-a3d2-7d41f042604a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:08.983296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7548d-6a37-4d52-a3d2-7d41f042604a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:08.983799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7548d-6a37-4d52-a3d2-7d41f042604a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:08.996307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a7548d-6a37-4d52-a3d2-7d41f042604a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-35a7548d-6a37-4d52-a3d2-7d41f042604a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:28:09.002059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:09.003041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 18 11:28:09.003892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:09.006739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 338/685] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Oct 18 11:28:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:09.013882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb1b9b-616f-4778-8bb9-7bb65adea95b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:09.041672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7548d-6a37-4d52-a3d2-7d41f042604a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:09.134520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:09.143464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb1b9b-616f-4778-8bb9-7bb65adea95b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:09.158692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7548d-6a37-4d52-a3d2-7d41f042604a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:09.191878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc254f-ea9c-4610-a7f2-30c37dad54a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:09.192747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc254f-ea9c-4610-a7f2-30c37dad54a8 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:09.193289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc254f-ea9c-4610-a7f2-30c37dad54a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:09.193762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc254f-ea9c-4610-a7f2-30c37dad54a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:09.194627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc254f-ea9c-4610-a7f2-30c37dad54a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:09.195127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc254f-ea9c-4610-a7f2-30c37dad54a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:09.196713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc254f-ea9c-4610-a7f2-30c37dad54a8 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:09.201485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c62535-d330-44a1-b296-e1705dce4697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:09.202121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c62535-d330-44a1-b296-e1705dce4697 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:09.202399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c62535-d330-44a1-b296-e1705dce4697 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:09.202671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c62535-d330-44a1-b296-e1705dce4697 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:09.203261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c62535-d330-44a1-b296-e1705dce4697 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:09.204743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c62535-d330-44a1-b296-e1705dce4697 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:09.212425 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 348/686] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Oct 18 11:28:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:09.213681 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc254f-ea9c-4610-a7f2-30c37dad54a8 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-e8c62535-d330-44a1-b296-e1705dce4697 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:09.214418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:09.215378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:09.216353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 339/687] 10.222.0.17 () {56 vars in 1132 bytes} [Tue Oct 18 11:28:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:09.221954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c62535-d330-44a1-b296-e1705dce4697 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:09.274941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:09.337901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc254f-ea9c-4610-a7f2-30c37dad54a8 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:09.357263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c62535-d330-44a1-b296-e1705dce4697 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:09.381999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f5d6e-0c4c-4522-a487-c7c5fa6b6e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:09.382638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5d6e-0c4c-4522-a487-c7c5fa6b6e42 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:09.382911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5d6e-0c4c-4522-a487-c7c5fa6b6e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:09.383183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f5d6e-0c4c-4522-a487-c7c5fa6b6e42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:09.396188 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04023b-428a-432e-8010-74231892ff94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:09.397249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04023b-428a-432e-8010-74231892ff94 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:09.397745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04023b-428a-432e-8010-74231892ff94 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:09.398116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04023b-428a-432e-8010-74231892ff94 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:09.398858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04023b-428a-432e-8010-74231892ff94 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:09.399176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f5d6e-0c4c-4522-a487-c7c5fa6b6e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:09.399332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04023b-428a-432e-8010-74231892ff94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:09.400803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04023b-428a-432e-8010-74231892ff94 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:09.401088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f5d6e-0c4c-4522-a487-c7c5fa6b6e42 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:09.410359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 340/688] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:28:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:09.412402 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04023b-428a-432e-8010-74231892ff94 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-da1f5d6e-0c4c-4522-a487-c7c5fa6b6e42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:28:09.422895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:09.423922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:28:09.424873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:28:09.424873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:09.424873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:09.428639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 349/689] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:09.429742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f5d6e-0c4c-4522-a487-c7c5fa6b6e42 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:09.432653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:09.541499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:09.545626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f5d6e-0c4c-4522-a487-c7c5fa6b6e42 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:09.546246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04023b-428a-432e-8010-74231892ff94 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:09.583582 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bacb1-de70-4126-8fe7-997c99832677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:09.584203 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bacb1-de70-4126-8fe7-997c99832677 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:09.584482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bacb1-de70-4126-8fe7-997c99832677 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:09.584752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bacb1-de70-4126-8fe7-997c99832677 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:09.585449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bacb1-de70-4126-8fe7-997c99832677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:09.585747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bacb1-de70-4126-8fe7-997c99832677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:09.587158 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bacb1-de70-4126-8fe7-997c99832677 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:09.589098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec57e6-7feb-4798-a1da-707788f15dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:09.590062 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec57e6-7feb-4798-a1da-707788f15dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:09.590647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec57e6-7feb-4798-a1da-707788f15dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:09.591144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec57e6-7feb-4798-a1da-707788f15dc5 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:09.592002 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec57e6-7feb-4798-a1da-707788f15dc5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:09.592505 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec57e6-7feb-4798-a1da-707788f15dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:09.594133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec57e6-7feb-4798-a1da-707788f15dc5 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:09.604913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 350/690] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:09.605110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 341/691] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:28:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:09.607592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bacb1-de70-4126-8fe7-997c99832677 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:09.608633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c681d-1e3c-4f89-9423-77a1e0545958 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:09.608960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c681d-1e3c-4f89-9423-77a1e0545958 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:09.609234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c681d-1e3c-4f89-9423-77a1e0545958 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:09.612174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 342/692] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:09.613651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c681d-1e3c-4f89-9423-77a1e0545958 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:09.691173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:09.721135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bacb1-de70-4126-8fe7-997c99832677 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:09.725058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c681d-1e3c-4f89-9423-77a1e0545958 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:09.759398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbc6f1-4172-4b20-8e1c-03bce3e93396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:09.760016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbc6f1-4172-4b20-8e1c-03bce3e93396 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:09.760288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbc6f1-4172-4b20-8e1c-03bce3e93396 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:09.760553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbc6f1-4172-4b20-8e1c-03bce3e93396 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:09.761294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbc6f1-4172-4b20-8e1c-03bce3e93396 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:09.762743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbc6f1-4172-4b20-8e1c-03bce3e93396 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:09.763126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148b21f-122e-4845-b848-2a9ade52f18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:09.763775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148b21f-122e-4845-b848-2a9ade52f18e None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:09.764055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148b21f-122e-4845-b848-2a9ade52f18e None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:09.764328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148b21f-122e-4845-b848-2a9ade52f18e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:09.773942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148b21f-122e-4845-b848-2a9ade52f18e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:09.775458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148b21f-122e-4845-b848-2a9ade52f18e None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:09.785933 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 351/693] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:28:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:28:09.787132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbae79-6df4-4cb0-b7b7-c5ceaca0d55e None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:09.787491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbae79-6df4-4cb0-b7b7-c5ceaca0d55e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:09.787772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbae79-6df4-4cb0-b7b7-c5ceaca0d55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-6148b21f-122e-4845-b848-2a9ade52f18e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:28:09.799327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:09.800368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:09.801345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:09.801345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:09.801345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:28:09.801345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:28:09.801345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:28:09.801345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:28:09.801345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:28:09.801345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:28:09.801345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 18 11:28:09.801345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:09.805683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 343/694] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Oct 18 11:28:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:09.809301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-30bbae79-6df4-4cb0-b7b7-c5ceaca0d55e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:09.811383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-30bbae79-6df4-4cb0-b7b7-c5ceaca0d55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:09.811930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148b21f-122e-4845-b848-2a9ade52f18e None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:09.824461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:09.907473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 352/695] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:09.967766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148b21f-122e-4845-b848-2a9ade52f18e None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:10.000520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:10.005913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1f18b-ed69-452f-9142-063ac337b755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:10.006526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1f18b-ed69-452f-9142-063ac337b755 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:10.006808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1f18b-ed69-452f-9142-063ac337b755 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:10.007081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1f18b-ed69-452f-9142-063ac337b755 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:10.007809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1f18b-ed69-452f-9142-063ac337b755 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:10.008115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1f18b-ed69-452f-9142-063ac337b755 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:10.009625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1f18b-ed69-452f-9142-063ac337b755 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:10.031771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 344/696] 10.222.0.17 () {56 vars in 1149 bytes} [Tue Oct 18 11:28:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:10.037912 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbae79-6df4-4cb0-b7b7-c5ceaca0d55e None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:10.147472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbae79-6df4-4cb0-b7b7-c5ceaca0d55e None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:10.185407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556e769-ec77-42fc-bf31-829f9c9ffa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:10.186024 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556e769-ec77-42fc-bf31-829f9c9ffa83 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:10.186290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556e769-ec77-42fc-bf31-829f9c9ffa83 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:10.186551 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556e769-ec77-42fc-bf31-829f9c9ffa83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:10.198176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:10.210973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1f18b-ed69-452f-9142-063ac337b755 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:10.265090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556e769-ec77-42fc-bf31-829f9c9ffa83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:10.266579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556e769-ec77-42fc-bf31-829f9c9ffa83 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:10.320499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1f18b-ed69-452f-9142-063ac337b755 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:10.359555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0affb-3c5c-48b3-90c5-61fa95e23db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:10.360611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0affb-3c5c-48b3-90c5-61fa95e23db7 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:10.361131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0affb-3c5c-48b3-90c5-61fa95e23db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:10.361609 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0affb-3c5c-48b3-90c5-61fa95e23db7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:10.362405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0affb-3c5c-48b3-90c5-61fa95e23db7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:10.364267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0affb-3c5c-48b3-90c5-61fa95e23db7 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-27d0affb-3c5c-48b3-90c5-61fa95e23db7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:10.374192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:10.375230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 18 11:28:10.376080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:10.382874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 353/697] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:28:10 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 18 11:28:10.383019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 345/698] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:28:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:10.385855 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556e769-ec77-42fc-bf31-829f9c9ffa83 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:10.389897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0affb-3c5c-48b3-90c5-61fa95e23db7 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:10.428488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:10.454900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:10.507804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0affb-3c5c-48b3-90c5-61fa95e23db7 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:10.509433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556e769-ec77-42fc-bf31-829f9c9ffa83 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:10.548339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ef9a2-7a21-4652-83f8-85a118a8ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:10.549363 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ef9a2-7a21-4652-83f8-85a118a8ad22 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:10.549918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ef9a2-7a21-4652-83f8-85a118a8ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:10.550390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ef9a2-7a21-4652-83f8-85a118a8ad22 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:10.551220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ef9a2-7a21-4652-83f8-85a118a8ad22 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:10.551759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ef9a2-7a21-4652-83f8-85a118a8ad22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:10.553586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ef9a2-7a21-4652-83f8-85a118a8ad22 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:10.556247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc6793-536a-4b13-b2c6-63505a4217d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:10.556917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc6793-536a-4b13-b2c6-63505a4217d0 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:10.557198 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc6793-536a-4b13-b2c6-63505a4217d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:10.557468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc6793-536a-4b13-b2c6-63505a4217d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:10.565152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 346/699] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:28:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:10.566401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ef9a2-7a21-4652-83f8-85a118a8ad22 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:10.568719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc6793-536a-4b13-b2c6-63505a4217d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:10.570333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc6793-536a-4b13-b2c6-63505a4217d0 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-19bc6793-536a-4b13-b2c6-63505a4217d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:28:10.587065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:10.588067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:28:10.589029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:28:10.589029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:10.589029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:10.589029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 354/700] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:10.595361 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc6793-536a-4b13-b2c6-63505a4217d0 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:10.702310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ef9a2-7a21-4652-83f8-85a118a8ad22 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:10.724054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:10.729226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:10.736830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc6793-536a-4b13-b2c6-63505a4217d0 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:10.754783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad292ac2-9f1f-4014-8c38-1b3bed48c83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:10.755354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad292ac2-9f1f-4014-8c38-1b3bed48c83f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:10.755548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad292ac2-9f1f-4014-8c38-1b3bed48c83f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:10.755804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad292ac2-9f1f-4014-8c38-1b3bed48c83f None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:10.756443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad292ac2-9f1f-4014-8c38-1b3bed48c83f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:10.756745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad292ac2-9f1f-4014-8c38-1b3bed48c83f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:10.759054 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad292ac2-9f1f-4014-8c38-1b3bed48c83f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:10.766140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 347/701] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:28:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:10.772257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad292ac2-9f1f-4014-8c38-1b3bed48c83f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:10.789565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab0079-0c0d-4513-ade9-ca5171e91465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:10.790913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab0079-0c0d-4513-ade9-ca5171e91465 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:10.792075 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab0079-0c0d-4513-ade9-ca5171e91465 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:10.792721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab0079-0c0d-4513-ade9-ca5171e91465 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:10.794497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab0079-0c0d-4513-ade9-ca5171e91465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:10.795001 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab0079-0c0d-4513-ade9-ca5171e91465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:10.797477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab0079-0c0d-4513-ade9-ca5171e91465 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:10.821211 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 355/702] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:10.822513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab0079-0c0d-4513-ade9-ca5171e91465 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:10.884122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad292ac2-9f1f-4014-8c38-1b3bed48c83f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:10.919351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b8687-6d6b-4b57-9a8f-441dff6dc05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:10.920056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b8687-6d6b-4b57-9a8f-441dff6dc05a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:10.920404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b8687-6d6b-4b57-9a8f-441dff6dc05a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:10.920747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b8687-6d6b-4b57-9a8f-441dff6dc05a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:10.921581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b8687-6d6b-4b57-9a8f-441dff6dc05a None admin] RBAC: Adding query filter params (interface=public, service_id=f2261b654d754119bb3d5992073135ab, region_id=RegionOne) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:10.921957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b8687-6d6b-4b57-9a8f-441dff6dc05a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:10.923399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b8687-6d6b-4b57-9a8f-441dff6dc05a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:10.930862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 348/703] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Oct 18 11:28:10 2022] GET /identity/v3/endpoints?service_id=f2261b654d754119bb3d5992073135ab&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:10.936762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b8687-6d6b-4b57-9a8f-441dff6dc05a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:10.958197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab0079-0c0d-4513-ade9-ca5171e91465 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:11.001302 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:11.004653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0ea89-e6ee-4296-b340-695ce1383a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:11.005745 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0ea89-e6ee-4296-b340-695ce1383a3e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:11.006128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0ea89-e6ee-4296-b340-695ce1383a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:11.006502 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0ea89-e6ee-4296-b340-695ce1383a3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:11.016626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0ea89-e6ee-4296-b340-695ce1383a3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:11.019118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0ea89-e6ee-4296-b340-695ce1383a3e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-2fb0ea89-e6ee-4296-b340-695ce1383a3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:28:11.034565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:11.035688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:11.036573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:28:11.036573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:28:11.036573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:28:11.036573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:28:11.036573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:28:11.036573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:28:11.036573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:11.036573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:11.037034 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 356/704] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:28:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:11.044552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0ea89-e6ee-4296-b340-695ce1383a3e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:11.060225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b8687-6d6b-4b57-9a8f-441dff6dc05a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:11.066222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:11.098163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefa672-c6f1-4708-8c0e-41863b3141a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:11.098783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefa672-c6f1-4708-8c0e-41863b3141a7 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:11.099065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefa672-c6f1-4708-8c0e-41863b3141a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:11.099340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefa672-c6f1-4708-8c0e-41863b3141a7 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:11.100018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefa672-c6f1-4708-8c0e-41863b3141a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:11.101539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefa672-c6f1-4708-8c0e-41863b3141a7 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:11.109077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 349/705] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Oct 18 11:28:10 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:28:11.173996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da8ee6-254c-4037-af75-4c79c967b968 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:11.174347 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da8ee6-254c-4037-af75-4c79c967b968 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:11.174549 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da8ee6-254c-4037-af75-4c79c967b968 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:11.174793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0ea89-e6ee-4296-b340-695ce1383a3e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:11.178394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 350/706] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:11.184892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0ee8d-72b5-43e3-b4e9-c6ac53c70c6c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:11.184892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0ee8d-72b5-43e3-b4e9-c6ac53c70c6c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:11.184892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0ee8d-72b5-43e3-b4e9-c6ac53c70c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:11.202777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-b0d0ee8d-72b5-43e3-b4e9-c6ac53c70c6c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:11.204023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b0d0ee8d-72b5-43e3-b4e9-c6ac53c70c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:11.224135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e47dfd-9e5d-42df-99b2-166fa3d92087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:11.224694 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e47dfd-9e5d-42df-99b2-166fa3d92087 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:11.225004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e47dfd-9e5d-42df-99b2-166fa3d92087 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:11.225278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e47dfd-9e5d-42df-99b2-166fa3d92087 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:11.225990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e47dfd-9e5d-42df-99b2-166fa3d92087 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:11.226296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e47dfd-9e5d-42df-99b2-166fa3d92087 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:11.227769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e47dfd-9e5d-42df-99b2-166fa3d92087 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:11.242427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 357/707] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:28:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:11.249947 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e47dfd-9e5d-42df-99b2-166fa3d92087 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:11.310424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 351/708] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:28:11.335498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:11.368374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e47dfd-9e5d-42df-99b2-166fa3d92087 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:11.404977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a723ab-a9e5-40da-bc44-b8bfb5ae42ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:11.405594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a723ab-a9e5-40da-bc44-b8bfb5ae42ab None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:11.405870 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a723ab-a9e5-40da-bc44-b8bfb5ae42ab None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:11.406138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a723ab-a9e5-40da-bc44-b8bfb5ae42ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:11.406863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a723ab-a9e5-40da-bc44-b8bfb5ae42ab None admin] RBAC: Adding query filter params (role.id=67d4f6145bce41c6a2168196bb6d8a74, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=74136098746a4c10b1a74051b75bbd2b) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:11.407166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a723ab-a9e5-40da-bc44-b8bfb5ae42ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:11.408587 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a723ab-a9e5-40da-bc44-b8bfb5ae42ab None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:11.415249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 358/709] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:28:11 2022] GET /identity/v3/role_assignments?user.id=74136098746a4c10b1a74051b75bbd2b&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=67d4f6145bce41c6a2168196bb6d8a74 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:11.599099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0ee8d-72b5-43e3-b4e9-c6ac53c70c6c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:11.645391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:11.709289 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0ee8d-72b5-43e3-b4e9-c6ac53c70c6c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:11.746621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1088344-e0fb-4943-8b6f-8d737067e3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:11.748340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1088344-e0fb-4943-8b6f-8d737067e3ce None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:11.748807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1088344-e0fb-4943-8b6f-8d737067e3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:11.749168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1088344-e0fb-4943-8b6f-8d737067e3ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:11.749775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1088344-e0fb-4943-8b6f-8d737067e3ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:11.751155 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1088344-e0fb-4943-8b6f-8d737067e3ce None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-b1088344-e0fb-4943-8b6f-8d737067e3ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:11.760608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:11.761687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:11.763105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 352/710] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:28:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:11.770260 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a723ab-a9e5-40da-bc44-b8bfb5ae42ab None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:11.878394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a723ab-a9e5-40da-bc44-b8bfb5ae42ab None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:11.915323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127fa39-c77c-44cf-99d0-9ab40c7869fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:11.916308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127fa39-c77c-44cf-99d0-9ab40c7869fa None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:11.916822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127fa39-c77c-44cf-99d0-9ab40c7869fa None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:11.917323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127fa39-c77c-44cf-99d0-9ab40c7869fa None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:11.918156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127fa39-c77c-44cf-99d0-9ab40c7869fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:11.918702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127fa39-c77c-44cf-99d0-9ab40c7869fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:11.920294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127fa39-c77c-44cf-99d0-9ab40c7869fa None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:11.922657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:11.926138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 359/711] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:28:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:11.931736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1088344-e0fb-4943-8b6f-8d737067e3ce None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:12.052923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1088344-e0fb-4943-8b6f-8d737067e3ce None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:12.056472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:12.093153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fe94f-5658-4b78-8b1a-46c97f93d264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:12.093797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fe94f-5658-4b78-8b1a-46c97f93d264 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:12.094078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fe94f-5658-4b78-8b1a-46c97f93d264 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:12.094352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fe94f-5658-4b78-8b1a-46c97f93d264 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:12.095002 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fe94f-5658-4b78-8b1a-46c97f93d264 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:12.095306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fe94f-5658-4b78-8b1a-46c97f93d264 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:12.096688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fe94f-5658-4b78-8b1a-46c97f93d264 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:12.099539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480cb46-f248-46a7-a384-3e5ec3f600ac None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:12.099822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480cb46-f248-46a7-a384-3e5ec3f600ac None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:12.100093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480cb46-f248-46a7-a384-3e5ec3f600ac None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:12.101725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 360/712] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:12.102671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 353/713] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:28:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:12.107076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c266eb-991c-4ed3-83f7-708ea14bcbd7 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:12.107413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c266eb-991c-4ed3-83f7-708ea14bcbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:12.107690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c266eb-991c-4ed3-83f7-708ea14bcbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:12.113127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fe94f-5658-4b78-8b1a-46c97f93d264 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:12.126657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-19c266eb-991c-4ed3-83f7-708ea14bcbd7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:12.127862 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-19c266eb-991c-4ed3-83f7-708ea14bcbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:12.215335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 361/714] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:12.231911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fe94f-5658-4b78-8b1a-46c97f93d264 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:12.268548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e07c1-ca25-46cf-b5cb-0d22c21aed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:12.269200 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e07c1-ca25-46cf-b5cb-0d22c21aed12 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:12.269529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e07c1-ca25-46cf-b5cb-0d22c21aed12 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:12.269804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e07c1-ca25-46cf-b5cb-0d22c21aed12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:12.270538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e07c1-ca25-46cf-b5cb-0d22c21aed12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:12.272024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e07c1-ca25-46cf-b5cb-0d22c21aed12 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:12.297334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 354/715] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:28:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:28:12.498179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c266eb-991c-4ed3-83f7-708ea14bcbd7 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:12.603657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c266eb-991c-4ed3-83f7-708ea14bcbd7 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:12.634235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:12.648364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6515c2-c0f2-4259-aab5-20b446030545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:12.649046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6515c2-c0f2-4259-aab5-20b446030545 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:12.649317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6515c2-c0f2-4259-aab5-20b446030545 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:12.649643 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6515c2-c0f2-4259-aab5-20b446030545 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:12.650220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6515c2-c0f2-4259-aab5-20b446030545 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:12.651566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6515c2-c0f2-4259-aab5-20b446030545 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-dc6515c2-c0f2-4259-aab5-20b446030545 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:12.661421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:12.662443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:12.663289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 362/716] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:28:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:12.670310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e07c1-ca25-46cf-b5cb-0d22c21aed12 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:12.696695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:12.779461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e07c1-ca25-46cf-b5cb-0d22c21aed12 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:12.815719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956f3e8-f5e9-4c2c-9dd0-3799625dfa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:12.816334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956f3e8-f5e9-4c2c-9dd0-3799625dfa8d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:12.816612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956f3e8-f5e9-4c2c-9dd0-3799625dfa8d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:12.816907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956f3e8-f5e9-4c2c-9dd0-3799625dfa8d None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:12.817546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956f3e8-f5e9-4c2c-9dd0-3799625dfa8d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:12.817846 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956f3e8-f5e9-4c2c-9dd0-3799625dfa8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:12.819206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956f3e8-f5e9-4c2c-9dd0-3799625dfa8d None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:12.824682 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 355/717] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:28:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:12.831015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6515c2-c0f2-4259-aab5-20b446030545 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:12.938672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6515c2-c0f2-4259-aab5-20b446030545 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:12.974502 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933138b-4f0a-4c3e-a2c2-6e1f7543c72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:12.975115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933138b-4f0a-4c3e-a2c2-6e1f7543c72e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:12.975390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933138b-4f0a-4c3e-a2c2-6e1f7543c72e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:12.975660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933138b-4f0a-4c3e-a2c2-6e1f7543c72e None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:12.976291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933138b-4f0a-4c3e-a2c2-6e1f7543c72e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:12.976594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933138b-4f0a-4c3e-a2c2-6e1f7543c72e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:12.977995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933138b-4f0a-4c3e-a2c2-6e1f7543c72e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:12.983193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 363/718] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:28:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:13.431541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384af14-bef1-414c-9651-b3972d2135a4 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:13.432045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384af14-bef1-414c-9651-b3972d2135a4 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:13.432045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384af14-bef1-414c-9651-b3972d2135a4 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:13.433678 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 356/719] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:13.439827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f564bc-a4c7-4243-8d46-fd375a44e820 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:13.440267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f564bc-a4c7-4243-8d46-fd375a44e820 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:13.440446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f564bc-a4c7-4243-8d46-fd375a44e820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:13.459543 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-37f564bc-a4c7-4243-8d46-fd375a44e820 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:13.460721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-37f564bc-a4c7-4243-8d46-fd375a44e820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:13.512760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:13.534179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 364/720] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:13.726387 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd11ed-ddab-4ff9-8d20-aa9ce5d881b2 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:13.726634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd11ed-ddab-4ff9-8d20-aa9ce5d881b2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:13.726862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd11ed-ddab-4ff9-8d20-aa9ce5d881b2 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:13.728451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 357/721] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:13.734711 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659d4b-11bb-4b88-bd41-c828f2e7bf9b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:13.734992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659d4b-11bb-4b88-bd41-c828f2e7bf9b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:13.735265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659d4b-11bb-4b88-bd41-c828f2e7bf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:13.754122 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-62659d4b-11bb-4b88-bd41-c828f2e7bf9b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:13.755318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-62659d4b-11bb-4b88-bd41-c828f2e7bf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:13.808633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd11ed-ddab-4ff9-8d20-aa9ce5d881b2 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:13.835888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 365/722] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:13.906808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:13.924405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd11ed-ddab-4ff9-8d20-aa9ce5d881b2 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:13.960162 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac906f-0d9b-4703-b037-93a9ae13315d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:13.961142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac906f-0d9b-4703-b037-93a9ae13315d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:13.961591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac906f-0d9b-4703-b037-93a9ae13315d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:13.962192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac906f-0d9b-4703-b037-93a9ae13315d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:13.962913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac906f-0d9b-4703-b037-93a9ae13315d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:13.964472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac906f-0d9b-4703-b037-93a9ae13315d None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-e5ac906f-0d9b-4703-b037-93a9ae13315d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:13.974513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:13.975658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 18 11:28:13.976749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:13.984225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 358/723] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:28:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:13.985997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62659d4b-11bb-4b88-bd41-c828f2e7bf9b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:14.095204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62659d4b-11bb-4b88-bd41-c828f2e7bf9b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:14.103269 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac906f-0d9b-4703-b037-93a9ae13315d None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:14.111950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:14.138328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ac179-117b-4457-bcd9-3852d3c8c15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:14.139760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac179-117b-4457-bcd9-3852d3c8c15e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:14.140330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac179-117b-4457-bcd9-3852d3c8c15e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:14.140745 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac179-117b-4457-bcd9-3852d3c8c15e None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:14.141533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ac179-117b-4457-bcd9-3852d3c8c15e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:14.141976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ac179-117b-4457-bcd9-3852d3c8c15e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:14.144980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ac179-117b-4457-bcd9-3852d3c8c15e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:14.156578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 366/724] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:28:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:14.158456 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ac179-117b-4457-bcd9-3852d3c8c15e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:14.233524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac906f-0d9b-4703-b037-93a9ae13315d None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:14.272214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e78305-7524-4a6d-baea-cd5e95255319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:14.273146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e78305-7524-4a6d-baea-cd5e95255319 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:14.273567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e78305-7524-4a6d-baea-cd5e95255319 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:14.273982 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e78305-7524-4a6d-baea-cd5e95255319 None admin] PATH_INFO: `/v3/services/image` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:14.274746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e78305-7524-4a6d-baea-cd5e95255319 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:14.276312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e78305-7524-4a6d-baea-cd5e95255319 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:14.278479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ac179-117b-4457-bcd9-3852d3c8c15e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-70e78305-7524-4a6d-baea-cd5e95255319 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:14.286695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:14.287743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 18 11:28:14.288586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:14.291610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 359/725] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Oct 18 11:28:14 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:14.299131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e78305-7524-4a6d-baea-cd5e95255319 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:14.318244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e74ba9-fc4c-4f38-a362-9f3f49f76dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:14.319374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74ba9-fc4c-4f38-a362-9f3f49f76dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:14.319659 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74ba9-fc4c-4f38-a362-9f3f49f76dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:14.319937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74ba9-fc4c-4f38-a362-9f3f49f76dc5 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:14.320884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e74ba9-fc4c-4f38-a362-9f3f49f76dc5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:14.321204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e74ba9-fc4c-4f38-a362-9f3f49f76dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:14.322675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:14.323527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e74ba9-fc4c-4f38-a362-9f3f49f76dc5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:14.331857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 367/726] 10.222.0.17 () {56 vars in 1114 bytes} [Tue Oct 18 11:28:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:14.345883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e74ba9-fc4c-4f38-a362-9f3f49f76dc5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:14.419713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:14.422226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e78305-7524-4a6d-baea-cd5e95255319 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:14.461585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b14078-046f-4dcb-a83b-a2682c905b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:14.462225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b14078-046f-4dcb-a83b-a2682c905b10 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:14.462479 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b14078-046f-4dcb-a83b-a2682c905b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:14.462722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b14078-046f-4dcb-a83b-a2682c905b10 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:14.463351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b14078-046f-4dcb-a83b-a2682c905b10 None admin] RBAC: Adding query filter params (name=image) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:14.463647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b14078-046f-4dcb-a83b-a2682c905b10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:14.465040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b14078-046f-4dcb-a83b-a2682c905b10 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:14.474886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 360/727] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:14.476232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b14078-046f-4dcb-a83b-a2682c905b10 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:14.499173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e74ba9-fc4c-4f38-a362-9f3f49f76dc5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:14.534309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b31c12-4ea6-4d1a-bd8d-9957cfc12bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:14.535151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31c12-4ea6-4d1a-bd8d-9957cfc12bd2 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:14.535286 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31c12-4ea6-4d1a-bd8d-9957cfc12bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:14.535472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31c12-4ea6-4d1a-bd8d-9957cfc12bd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:14.536103 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b31c12-4ea6-4d1a-bd8d-9957cfc12bd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:14.537503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b31c12-4ea6-4d1a-bd8d-9957cfc12bd2 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:14.565033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 368/728] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:28:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:14.578042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:14.604223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b14078-046f-4dcb-a83b-a2682c905b10 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:14.654355 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba2d59-0996-48eb-9129-c9588dc4ea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:14.655355 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba2d59-0996-48eb-9129-c9588dc4ea45 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:14.655858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba2d59-0996-48eb-9129-c9588dc4ea45 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:14.656354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba2d59-0996-48eb-9129-c9588dc4ea45 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:14.657286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba2d59-0996-48eb-9129-c9588dc4ea45 None admin] RBAC: Adding query filter params (type=image) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:14.657796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba2d59-0996-48eb-9129-c9588dc4ea45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:14.659408 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba2d59-0996-48eb-9129-c9588dc4ea45 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:14.665773 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 361/729] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:14.712405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a53beb-0066-4bf6-886e-b6052d15cf86 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:14.712707 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a53beb-0066-4bf6-886e-b6052d15cf86 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:14.713004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a53beb-0066-4bf6-886e-b6052d15cf86 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:14.714500 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 369/730] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:14.720669 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e2792-70fd-4598-839c-a0583b39369f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:14.721291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e2792-70fd-4598-839c-a0583b39369f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:14.721821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e2792-70fd-4598-839c-a0583b39369f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:14.743412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d41e2792-70fd-4598-839c-a0583b39369f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:14.745122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d41e2792-70fd-4598-839c-a0583b39369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:14.835974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 362/731] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:15.105785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a53beb-0066-4bf6-886e-b6052d15cf86 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:15.228219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:15.236170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a53beb-0066-4bf6-886e-b6052d15cf86 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:15.274644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d23c5-f5a7-43ce-bf8d-49cc7317c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:15.275332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d23c5-f5a7-43ce-bf8d-49cc7317c780 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:15.275607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d23c5-f5a7-43ce-bf8d-49cc7317c780 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:15.275873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d23c5-f5a7-43ce-bf8d-49cc7317c780 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:15.276447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d23c5-f5a7-43ce-bf8d-49cc7317c780 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:15.278507 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d23c5-f5a7-43ce-bf8d-49cc7317c780 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:15.285507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b07114-cf74-4881-96bd-ed8133af9973 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:15.285788 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b07114-cf74-4881-96bd-ed8133af9973 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:15.286067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b07114-cf74-4881-96bd-ed8133af9973 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:15.287646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 363/732] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-ca8d23c5-f5a7-43ce-bf8d-49cc7317c780 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:15.288311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:15.289305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:15.290176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 370/733] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:28:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:15.293328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89729e4-b969-41e4-9807-804a4dbc5a98 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:15.293328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89729e4-b969-41e4-9807-804a4dbc5a98 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:15.294214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89729e4-b969-41e4-9807-804a4dbc5a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:15.298304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d23c5-f5a7-43ce-bf8d-49cc7317c780 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:15.313032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d89729e4-b969-41e4-9807-804a4dbc5a98 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:15.314285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d89729e4-b969-41e4-9807-804a4dbc5a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:15.382508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:15.393132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 364/734] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:15.426998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d23c5-f5a7-43ce-bf8d-49cc7317c780 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:15.466984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e240d6-4968-4c76-b1d3-60666a76f7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:15.467610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e240d6-4968-4c76-b1d3-60666a76f7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:15.467888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e240d6-4968-4c76-b1d3-60666a76f7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:15.468162 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e240d6-4968-4c76-b1d3-60666a76f7d8 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:15.468795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e240d6-4968-4c76-b1d3-60666a76f7d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:15.469149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e240d6-4968-4c76-b1d3-60666a76f7d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:15.470607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e240d6-4968-4c76-b1d3-60666a76f7d8 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:15.475927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 371/735] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:28:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:15.482461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89729e4-b969-41e4-9807-804a4dbc5a98 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:15.591173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89729e4-b969-41e4-9807-804a4dbc5a98 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:15.633401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2fdc5-5c03-4af2-b1b6-db93aaea0406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:15.634379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2fdc5-5c03-4af2-b1b6-db93aaea0406 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:15.634875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2fdc5-5c03-4af2-b1b6-db93aaea0406 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:15.635342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2fdc5-5c03-4af2-b1b6-db93aaea0406 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:15.636182 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2fdc5-5c03-4af2-b1b6-db93aaea0406 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:15.636696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2fdc5-5c03-4af2-b1b6-db93aaea0406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:15.638344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2fdc5-5c03-4af2-b1b6-db93aaea0406 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:15.644435 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 365/736] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:28:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:15.650546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e240d6-4968-4c76-b1d3-60666a76f7d8 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:15.661422 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2fdc5-5c03-4af2-b1b6-db93aaea0406 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:15.686972 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:15.735916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:15.778262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2fdc5-5c03-4af2-b1b6-db93aaea0406 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:15.787866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e240d6-4968-4c76-b1d3-60666a76f7d8 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:15.823369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63a3f4-c2f2-4d80-b894-65af7133d3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:15.823988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a3f4-c2f2-4d80-b894-65af7133d3bd None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:15.824268 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a3f4-c2f2-4d80-b894-65af7133d3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:15.824540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a3f4-c2f2-4d80-b894-65af7133d3bd None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:15.825403 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63a3f4-c2f2-4d80-b894-65af7133d3bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:15.826764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63a3f4-c2f2-4d80-b894-65af7133d3bd None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:15.828692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712c89b-6db8-45df-a64a-fec34b157f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:15.829578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712c89b-6db8-45df-a64a-fec34b157f82 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:15.830164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712c89b-6db8-45df-a64a-fec34b157f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:15.830638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712c89b-6db8-45df-a64a-fec34b157f82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:15.831669 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712c89b-6db8-45df-a64a-fec34b157f82 None admin] RBAC: Adding query filter params (interface=admin, service_id=25684b799fa643609d07e8686d115576, region_id=RegionOne) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:15.832232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712c89b-6db8-45df-a64a-fec34b157f82 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:15.833821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712c89b-6db8-45df-a64a-fec34b157f82 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:15.844461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 372/737] 10.222.0.17 () {56 vars in 1249 bytes} [Tue Oct 18 11:28:15 2022] GET /identity/v3/endpoints?service_id=25684b799fa643609d07e8686d115576&interface=admin®ion_id=RegionOne => generated 193 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:15.845747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712c89b-6db8-45df-a64a-fec34b157f82 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:15.860948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 366/738] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:28:15 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:15.967912 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:15.979673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712c89b-6db8-45df-a64a-fec34b157f82 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:16.028745 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d39f8-7d2a-4f2e-a26c-babbe83dc24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:16.029807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d39f8-7d2a-4f2e-a26c-babbe83dc24c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:16.030310 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d39f8-7d2a-4f2e-a26c-babbe83dc24c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:16.030837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d39f8-7d2a-4f2e-a26c-babbe83dc24c None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:16.031743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d39f8-7d2a-4f2e-a26c-babbe83dc24c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:16.033533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d39f8-7d2a-4f2e-a26c-babbe83dc24c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:16.041236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 373/739] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Oct 18 11:28:15 2022] GET /identity/v3/services => generated 1727 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:28:16.919805 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19187b-4761-41b1-9c0b-fbf645acef24 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:16.920367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19187b-4761-41b1-9c0b-fbf645acef24 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:16.920367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19187b-4761-41b1-9c0b-fbf645acef24 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:16.921938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 367/740] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:16.927832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83abb8-ff09-4d0e-b5a4-2e1a12b94d1e None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:16.928369 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83abb8-ff09-4d0e-b5a4-2e1a12b94d1e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:16.928369 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83abb8-ff09-4d0e-b5a4-2e1a12b94d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:16.948234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3f83abb8-ff09-4d0e-b5a4-2e1a12b94d1e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:16.950463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3f83abb8-ff09-4d0e-b5a4-2e1a12b94d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:16.962423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0d06e-e533-485d-aea6-ec0eb2fa1c69 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:16.962756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0d06e-e533-485d-aea6-ec0eb2fa1c69 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:16.962964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0d06e-e533-485d-aea6-ec0eb2fa1c69 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:16.964731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 368/741] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:16.970920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3675f-9b35-4eaa-9c56-51c5e76150dd None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:16.971314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3675f-9b35-4eaa-9c56-51c5e76150dd None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:16.971487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3675f-9b35-4eaa-9c56-51c5e76150dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:16.991516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e8a3675f-9b35-4eaa-9c56-51c5e76150dd None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:16.992733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e8a3675f-9b35-4eaa-9c56-51c5e76150dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:17.042834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 374/742] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:17.088978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 369/743] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:17.308504 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83abb8-ff09-4d0e-b5a4-2e1a12b94d1e None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:17.352174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3675f-9b35-4eaa-9c56-51c5e76150dd None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:17.406834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:17.412209 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:17.436966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83abb8-ff09-4d0e-b5a4-2e1a12b94d1e None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:17.475578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3675f-9b35-4eaa-9c56-51c5e76150dd None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:17.482379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:17.483005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:17.483286 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:17.483558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:17.484222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:17.485832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:17.488971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WARNING keystone.server.flask.common [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 18 11:28:17.510149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.sql.core [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 18 11:28:17.510149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 18 11:28:17.510149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [parameters: {'id': 'a23f569f18c34d5fa008dd60608aa7b2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 18 11:28:17.510149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81701) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 18 11:28:17.518561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809dd4e-f893-4414-bf49-f50d68acd0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:17.519173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809dd4e-f893-4414-bf49-f50d68acd0e5 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:17.519450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809dd4e-f893-4414-bf49-f50d68acd0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:17.519718 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809dd4e-f893-4414-bf49-f50d68acd0e5 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a23f569f18c34d5fa008dd60608aa7b2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a23f569f18c34d5fa008dd60608aa7b2. Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result = self._query(query) Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application conn.query(q) Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 18 11:28:17.520287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result.read() Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 18 11:28:17.521134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application next(self.gen) Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application yield resource Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application next(self.gen) Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.session.rollback() Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.force_reraise() Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise self.value Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application session.commit() Oct 18 11:28:17.522100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._prepare_impl() Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.session.flush() Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._flush(objects) Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application compat.raise_( Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application flush_context.execute() Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 18 11:28:17.522997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rec.execute(self) Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 18 11:28:17.523906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result = self._query(query) Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application conn.query(q) Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result.read() Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 18 11:28:17.524725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [parameters: {'id': 'a23f569f18c34d5fa008dd60608aa7b2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:17.527489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:17.534572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:17.536758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:17.536758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 18 11:28:17.536758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 18 11:28:17.536758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:17.536758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:17.536758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 18 11:28:17.536758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 18 11:28:17.536758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a23f569f18c34d5fa008dd60608aa7b2. Oct 18 11:28:17.536758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:17.537922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809dd4e-f893-4414-bf49-f50d68acd0e5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:17.538067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 375/744] 10.222.0.17 () {60 vars in 1140 bytes} [Tue Oct 18 11:28:17 2022] POST /identity/v3/projects => generated 262 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:17.538205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809dd4e-f893-4414-bf49-f50d68acd0e5 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:17.538344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:17.545000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 370/745] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:28:17 2022] POST /identity/v3/services => generated 242 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:17.683521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c2e27-72fc-40fc-800c-a15a82a1cce5 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:17.732216 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:17.738909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08222a07-970b-4ad0-a174-162795caa8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:17.739910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08222a07-970b-4ad0-a174-162795caa8ab None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:17.740476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08222a07-970b-4ad0-a174-162795caa8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:17.740960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08222a07-970b-4ad0-a174-162795caa8ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:17.751144 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08222a07-970b-4ad0-a174-162795caa8ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:17.753072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08222a07-970b-4ad0-a174-162795caa8ab None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-08222a07-970b-4ad0-a174-162795caa8ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:28:17.765831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:17.766878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:17.767961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:28:17.767961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:28:17.767961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:28:17.767961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:28:17.767961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:28:17.767961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:28:17.767961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:17.767961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:17.772912 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 376/746] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:28:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:17.775670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809dd4e-f893-4414-bf49-f50d68acd0e5 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:17.884367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809dd4e-f893-4414-bf49-f50d68acd0e5 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:17.899458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:17.923899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a52bb-9a30-4ceb-972d-708d23f4fc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:17.925223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a52bb-9a30-4ceb-972d-708d23f4fc9f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:17.925678 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a52bb-9a30-4ceb-972d-708d23f4fc9f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:17.926092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a52bb-9a30-4ceb-972d-708d23f4fc9f None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:17.926920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a52bb-9a30-4ceb-972d-708d23f4fc9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:17.927368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a52bb-9a30-4ceb-972d-708d23f4fc9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:17.928989 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a52bb-9a30-4ceb-972d-708d23f4fc9f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:17.947608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 371/747] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:28:17 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:18.182698 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12f858-8189-4676-b7bd-1dc5b6c09597 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:18.183199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12f858-8189-4676-b7bd-1dc5b6c09597 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:18.183281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12f858-8189-4676-b7bd-1dc5b6c09597 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:18.185346 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 377/748] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:18.191133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86533b-800d-4670-87d5-6b7ec8d0c420 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:18.191704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86533b-800d-4670-87d5-6b7ec8d0c420 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:18.192183 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86533b-800d-4670-87d5-6b7ec8d0c420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:18.212920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-0f86533b-800d-4670-87d5-6b7ec8d0c420 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:18.214585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0f86533b-800d-4670-87d5-6b7ec8d0c420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:18.292347 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 372/749] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:18 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:18.327339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d853f-a385-4929-9a86-cea4c909dbc2 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:18.327673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d853f-a385-4929-9a86-cea4c909dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:18.327870 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d853f-a385-4929-9a86-cea4c909dbc2 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:18.329503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 378/750] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:18.334959 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e086ea-035c-4332-a921-33b5f1e8de11 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:18.335247 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e086ea-035c-4332-a921-33b5f1e8de11 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:18.335524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e086ea-035c-4332-a921-33b5f1e8de11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:18.354454 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c4e086ea-035c-4332-a921-33b5f1e8de11 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:18.355628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c4e086ea-035c-4332-a921-33b5f1e8de11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:18.430823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 373/751] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:18 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:18.562914 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d853f-a385-4929-9a86-cea4c909dbc2 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:18.674355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d853f-a385-4929-9a86-cea4c909dbc2 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:18.708561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e086ea-035c-4332-a921-33b5f1e8de11 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:18.710397 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352690a2-d57b-45ee-9868-1fde9fedbf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:18.711233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352690a2-d57b-45ee-9868-1fde9fedbf57 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:18.711512 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352690a2-d57b-45ee-9868-1fde9fedbf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:18.712686 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352690a2-d57b-45ee-9868-1fde9fedbf57 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:18.713317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352690a2-d57b-45ee-9868-1fde9fedbf57 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:18.714692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352690a2-d57b-45ee-9868-1fde9fedbf57 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-352690a2-d57b-45ee-9868-1fde9fedbf57 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:18.726974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:18.728175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:18.729249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 379/752] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:28:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:18.736201 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352690a2-d57b-45ee-9868-1fde9fedbf57 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:18.754338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:18.825231 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:18.829788 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e086ea-035c-4332-a921-33b5f1e8de11 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:18.858031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352690a2-d57b-45ee-9868-1fde9fedbf57 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:18.866195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9643348-149f-41b1-8f49-7d017e330524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:18.866924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9643348-149f-41b1-8f49-7d017e330524 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:18.867271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9643348-149f-41b1-8f49-7d017e330524 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:18.867639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9643348-149f-41b1-8f49-7d017e330524 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:18.868302 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9643348-149f-41b1-8f49-7d017e330524 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:18.869817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9643348-149f-41b1-8f49-7d017e330524 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-b9643348-149f-41b1-8f49-7d017e330524 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:18.879736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:18.880752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:18.881808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 374/753] 10.222.0.17 () {56 vars in 1126 bytes} [Tue Oct 18 11:28:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:18.888797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9643348-149f-41b1-8f49-7d017e330524 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:18.903921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba706e-a43e-4499-a4a3-c9d98d150a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:18.904875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba706e-a43e-4499-a4a3-c9d98d150a29 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:18.905390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba706e-a43e-4499-a4a3-c9d98d150a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:18.905854 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba706e-a43e-4499-a4a3-c9d98d150a29 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:18.906791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba706e-a43e-4499-a4a3-c9d98d150a29 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:18.907384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba706e-a43e-4499-a4a3-c9d98d150a29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:18.908955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba706e-a43e-4499-a4a3-c9d98d150a29 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:18.915076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 380/754] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:28:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:18.923924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba706e-a43e-4499-a4a3-c9d98d150a29 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:19.007365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:19.011638 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9643348-149f-41b1-8f49-7d017e330524 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:19.046686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd73c3-57df-4d57-924c-c444536fc566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:19.047319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd73c3-57df-4d57-924c-c444536fc566 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:19.047670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd73c3-57df-4d57-924c-c444536fc566 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:19.047953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd73c3-57df-4d57-924c-c444536fc566 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:19.048723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd73c3-57df-4d57-924c-c444536fc566 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:19.048915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd73c3-57df-4d57-924c-c444536fc566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:19.050466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd73c3-57df-4d57-924c-c444536fc566 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:19.055671 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba706e-a43e-4499-a4a3-c9d98d150a29 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:19.055819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 375/755] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:28:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:19.064897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd73c3-57df-4d57-924c-c444536fc566 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:19.077921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:19.093383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a379fa-d3c5-488e-8800-c0ee9009d61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:19.094293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a379fa-d3c5-488e-8800-c0ee9009d61c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:19.094748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a379fa-d3c5-488e-8800-c0ee9009d61c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:19.095223 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a379fa-d3c5-488e-8800-c0ee9009d61c None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:19.096028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a379fa-d3c5-488e-8800-c0ee9009d61c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:19.096503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a379fa-d3c5-488e-8800-c0ee9009d61c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:19.098076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a379fa-d3c5-488e-8800-c0ee9009d61c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:19.103729 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 381/756] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:28:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:19.116351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a379fa-d3c5-488e-8800-c0ee9009d61c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:19.192621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd73c3-57df-4d57-924c-c444536fc566 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:19.229005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9001aba-4d7a-485a-ba38-6ceeb660c4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:19.229715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9001aba-4d7a-485a-ba38-6ceeb660c4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:19.230001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9001aba-4d7a-485a-ba38-6ceeb660c4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:19.230304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9001aba-4d7a-485a-ba38-6ceeb660c4d6 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:19.231496 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9001aba-4d7a-485a-ba38-6ceeb660c4d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:19.231794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9001aba-4d7a-485a-ba38-6ceeb660c4d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:19.233250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9001aba-4d7a-485a-ba38-6ceeb660c4d6 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:19.243495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 376/757] 10.222.0.17 () {56 vars in 1122 bytes} [Tue Oct 18 11:28:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:19.244808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9001aba-4d7a-485a-ba38-6ceeb660c4d6 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:19.267719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a379fa-d3c5-488e-8800-c0ee9009d61c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:19.279896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:19.317948 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828c089-03ad-46c9-aac7-f37f814883ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:19.318904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828c089-03ad-46c9-aac7-f37f814883ab None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:19.319646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828c089-03ad-46c9-aac7-f37f814883ab None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:19.320490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828c089-03ad-46c9-aac7-f37f814883ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:19.321758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828c089-03ad-46c9-aac7-f37f814883ab None admin] RBAC: Adding query filter params (interface=public, service_id=d688f25d5849461a8ecba170ee351c81, region_id=RegionOne) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:19.322220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828c089-03ad-46c9-aac7-f37f814883ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:19.325257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828c089-03ad-46c9-aac7-f37f814883ab None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:19.337230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 382/758] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Oct 18 11:28:19 2022] GET /identity/v3/endpoints?service_id=d688f25d5849461a8ecba170ee351c81&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:19.338943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828c089-03ad-46c9-aac7-f37f814883ab None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:19.379585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9001aba-4d7a-485a-ba38-6ceeb660c4d6 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:19.392580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:19.421718 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29dbf1-41f9-4f3f-9b8c-6d53b1760d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:19.422728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29dbf1-41f9-4f3f-9b8c-6d53b1760d38 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:19.423233 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29dbf1-41f9-4f3f-9b8c-6d53b1760d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:19.423706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29dbf1-41f9-4f3f-9b8c-6d53b1760d38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:19.424591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29dbf1-41f9-4f3f-9b8c-6d53b1760d38 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:19.426189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29dbf1-41f9-4f3f-9b8c-6d53b1760d38 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:19.450547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 377/759] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:28:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:19.453280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828c089-03ad-46c9-aac7-f37f814883ab None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:19.506352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba0eef-0447-43f6-bb7e-1940c57c7299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:19.507052 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba0eef-0447-43f6-bb7e-1940c57c7299 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:19.507336 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba0eef-0447-43f6-bb7e-1940c57c7299 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:19.507610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba0eef-0447-43f6-bb7e-1940c57c7299 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:19.508237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba0eef-0447-43f6-bb7e-1940c57c7299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:19.509796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba0eef-0447-43f6-bb7e-1940c57c7299 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:19.518573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 383/760] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Oct 18 11:28:19 2022] GET /identity/v3/services => generated 1957 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:28:19.964491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f470d-9149-4b54-903b-f0c3b17f40d7 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:19.964913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f470d-9149-4b54-903b-f0c3b17f40d7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:19.965628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f470d-9149-4b54-903b-f0c3b17f40d7 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:19.967211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 378/761] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:19.972611 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5c36d-870e-4ac4-9c61-1489b7934729 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:19.972933 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5c36d-870e-4ac4-9c61-1489b7934729 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:19.973210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5c36d-870e-4ac4-9c61-1489b7934729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:19.993077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e5a5c36d-870e-4ac4-9c61-1489b7934729 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:19.994590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e5a5c36d-870e-4ac4-9c61-1489b7934729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:20.073175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 384/762] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:19 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:20.283240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0bc36-fe92-4312-bb88-76bce6114caf None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:20.283759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0bc36-fe92-4312-bb88-76bce6114caf None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:20.283980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0bc36-fe92-4312-bb88-76bce6114caf None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:20.285759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 379/763] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:20.291323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3edba-177b-47e3-a65c-96755ebed4bb None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:20.291799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3edba-177b-47e3-a65c-96755ebed4bb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:20.292096 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3edba-177b-47e3-a65c-96755ebed4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:20.311365 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-58e3edba-177b-47e3-a65c-96755ebed4bb None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:20.312564 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-58e3edba-177b-47e3-a65c-96755ebed4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:20.344508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0bc36-fe92-4312-bb88-76bce6114caf None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:20.345636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:20.386513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 385/764] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:20 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:20.445266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:20.458415 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0bc36-fe92-4312-bb88-76bce6114caf None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:20.497743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5b774-3b1b-4aa0-940c-15e57ab242a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:20.498389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5b774-3b1b-4aa0-940c-15e57ab242a2 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:20.498669 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5b774-3b1b-4aa0-940c-15e57ab242a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:20.498933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5b774-3b1b-4aa0-940c-15e57ab242a2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:20.499551 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5b774-3b1b-4aa0-940c-15e57ab242a2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:20.500935 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5b774-3b1b-4aa0-940c-15e57ab242a2 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-4ad5b774-3b1b-4aa0-940c-15e57ab242a2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:20.509988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:20.510961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:20.511800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 380/765] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Oct 18 11:28:20 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:20.522208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3edba-177b-47e3-a65c-96755ebed4bb None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:20.626292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3edba-177b-47e3-a65c-96755ebed4bb None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:20.661966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57dff2-023e-4a86-a0a7-5d0fca779f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:20.662942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57dff2-023e-4a86-a0a7-5d0fca779f6b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:20.663419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57dff2-023e-4a86-a0a7-5d0fca779f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:20.663888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57dff2-023e-4a86-a0a7-5d0fca779f6b None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:20.664705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57dff2-023e-4a86-a0a7-5d0fca779f6b None admin] RBAC: Adding query filter params (name=image) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:20.665245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57dff2-023e-4a86-a0a7-5d0fca779f6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:20.667043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57dff2-023e-4a86-a0a7-5d0fca779f6b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:20.677432 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 386/766] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:20.683517 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5b774-3b1b-4aa0-940c-15e57ab242a2 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:20.692151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57dff2-023e-4a86-a0a7-5d0fca779f6b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:20.725213 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:20.803295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57dff2-023e-4a86-a0a7-5d0fca779f6b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:20.807491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5b774-3b1b-4aa0-940c-15e57ab242a2 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:20.842857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c5127-dfe7-4d0e-b973-fd9e6a0148a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:20.843729 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c5127-dfe7-4d0e-b973-fd9e6a0148a3 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:20.844121 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c5127-dfe7-4d0e-b973-fd9e6a0148a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:20.844460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c5127-dfe7-4d0e-b973-fd9e6a0148a3 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:20.845330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c5127-dfe7-4d0e-b973-fd9e6a0148a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:20.846843 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c5127-dfe7-4d0e-b973-fd9e6a0148a3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:20.850503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f8f23-cfe1-4295-99f1-98f79857532f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:20.851355 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f8f23-cfe1-4295-99f1-98f79857532f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:20.851858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f8f23-cfe1-4295-99f1-98f79857532f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:20.852309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f8f23-cfe1-4295-99f1-98f79857532f None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:20.853160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f8f23-cfe1-4295-99f1-98f79857532f None admin] RBAC: Adding query filter params (type=image) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:20.853642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f8f23-cfe1-4295-99f1-98f79857532f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:20.855174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f8f23-cfe1-4295-99f1-98f79857532f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:20.862813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 381/767] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:20.866244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.sql.core [None req-285c5127-dfe7-4d0e-b973-fd9e6a0148a3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 18 11:28:20.866244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 18 11:28:20.866244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [parameters: {'user_id': '085c2da95df5434da90ba71683505a0d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 18 11:28:20.866244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81701) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 18 11:28:20.869844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f8f23-cfe1-4295-99f1-98f79857532f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-285c5127-dfe7-4d0e-b973-fd9e6a0148a3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result = self._query(query) Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application conn.query(q) Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result.read() Oct 18 11:28:20.876644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 18 11:28:20.877566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application next(self.gen) Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application yield resource Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application next(self.gen) Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.session.rollback() Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.force_reraise() Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise self.value Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application session.commit() Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 18 11:28:20.878518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._prepare_impl() Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.session.flush() Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._flush(objects) Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application compat.raise_( Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application flush_context.execute() Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rec.execute(self) Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 18 11:28:20.879481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 18 11:28:20.880374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result = self._query(query) Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application conn.query(q) Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result.read() Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 18 11:28:20.888061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [parameters: {'user_id': '085c2da95df5434da90ba71683505a0d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:20.890090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 18 11:28:20.892023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 18 11:28:20.894195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 18 11:28:20.894195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 18 11:28:20.894195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 18 11:28:20.894195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 18 11:28:20.894195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 18 11:28:20.894195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:20.894195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 387/768] 10.222.0.17 () {60 vars in 1131 bytes} [Tue Oct 18 11:28:20 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:20.894195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c5127-dfe7-4d0e-b973-fd9e6a0148a3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:20.901382 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:21.001175 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f8f23-cfe1-4295-99f1-98f79857532f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:21.040691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c83b5e-e09d-4c9c-bd78-0a7a5ede7aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:21.041338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c83b5e-e09d-4c9c-bd78-0a7a5ede7aab None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:21.041616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c83b5e-e09d-4c9c-bd78-0a7a5ede7aab None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:21.041890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c83b5e-e09d-4c9c-bd78-0a7a5ede7aab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:21.042597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c83b5e-e09d-4c9c-bd78-0a7a5ede7aab None admin] RBAC: Adding query filter params (interface=public, service_id=cb2705dc26cb4e74823a568920012ac6, region_id=RegionOne) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:21.042918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c83b5e-e09d-4c9c-bd78-0a7a5ede7aab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:21.044831 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c83b5e-e09d-4c9c-bd78-0a7a5ede7aab None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:21.056598 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 382/769] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Oct 18 11:28:20 2022] GET /identity/v3/endpoints?service_id=cb2705dc26cb4e74823a568920012ac6&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:21.057824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c83b5e-e09d-4c9c-bd78-0a7a5ede7aab None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:21.140080 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:21.149489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c5127-dfe7-4d0e-b973-fd9e6a0148a3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:21.149723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:21.174438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c83b5e-e09d-4c9c-bd78-0a7a5ede7aab None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:21.194932 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988678fd-6eae-4ed6-b5a2-2d7fefd38013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:21.195988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988678fd-6eae-4ed6-b5a2-2d7fefd38013 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:21.196267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988678fd-6eae-4ed6-b5a2-2d7fefd38013 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:21.196538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988678fd-6eae-4ed6-b5a2-2d7fefd38013 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:21.207280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988678fd-6eae-4ed6-b5a2-2d7fefd38013 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:21.209571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988678fd-6eae-4ed6-b5a2-2d7fefd38013 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:21.216142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76186398-d5c7-46a0-87e7-0b454fd307cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:21.217101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186398-d5c7-46a0-87e7-0b454fd307cb None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:21.217584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186398-d5c7-46a0-87e7-0b454fd307cb None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:21.218219 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186398-d5c7-46a0-87e7-0b454fd307cb None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:21.219113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76186398-d5c7-46a0-87e7-0b454fd307cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:21.220746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76186398-d5c7-46a0-87e7-0b454fd307cb None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-988678fd-6eae-4ed6-b5a2-2d7fefd38013 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:28:21.224754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:21.225690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:21.226624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:21.226624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:21.226624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:28:21.226624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:28:21.226624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:28:21.226624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:28:21.226624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:28:21.226624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:28:21.226624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 18 11:28:21.226624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:21.236886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 388/770] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:28:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:21.236886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988678fd-6eae-4ed6-b5a2-2d7fefd38013 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:21.237193 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 383/771] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Oct 18 11:28:21 2022] GET /identity/v3/services => generated 1957 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:28:21.358322 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988678fd-6eae-4ed6-b5a2-2d7fefd38013 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:21.397345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91929a56-292f-4aa4-9add-5da10061ca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:21.397897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929a56-292f-4aa4-9add-5da10061ca51 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:21.398176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929a56-292f-4aa4-9add-5da10061ca51 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:21.398447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929a56-292f-4aa4-9add-5da10061ca51 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:21.399165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929a56-292f-4aa4-9add-5da10061ca51 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:21.399463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929a56-292f-4aa4-9add-5da10061ca51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:21.400939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929a56-292f-4aa4-9add-5da10061ca51 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:21.446195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 389/772] 10.222.0.17 () {56 vars in 1101 bytes} [Tue Oct 18 11:28:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:21.703999 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39a039-639b-4a52-af92-904b58b6a641 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:21.704479 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39a039-639b-4a52-af92-904b58b6a641 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:21.704479 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39a039-639b-4a52-af92-904b58b6a641 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:21.706167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 384/773] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:21.712723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160f55b-75aa-4521-824a-f7892a295bf5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:21.713092 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160f55b-75aa-4521-824a-f7892a295bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:21.713321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160f55b-75aa-4521-824a-f7892a295bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:21.734166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-f160f55b-75aa-4521-824a-f7892a295bf5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:21.735773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f160f55b-75aa-4521-824a-f7892a295bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:21.745085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:21.813201 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 390/774] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:21 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:21.834987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fd18c-1461-4f9d-af77-b57aba2828ae None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:21.835348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fd18c-1461-4f9d-af77-b57aba2828ae None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:21.835691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fd18c-1461-4f9d-af77-b57aba2828ae None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:21.837257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 385/775] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:21.843049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bfd6-fc5e-4219-bcb9-e987d76f308c None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:21.843338 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bfd6-fc5e-4219-bcb9-e987d76f308c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:21.843609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bfd6-fc5e-4219-bcb9-e987d76f308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:21.861651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9763bfd6-fc5e-4219-bcb9-e987d76f308c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:21.862835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9763bfd6-fc5e-4219-bcb9-e987d76f308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:21.930197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 391/776] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:21 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:22.080526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fd18c-1461-4f9d-af77-b57aba2828ae None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:22.192219 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fd18c-1461-4f9d-af77-b57aba2828ae None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:22.204155 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763bfd6-fc5e-4219-bcb9-e987d76f308c None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:22.228258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa7854-42bb-4383-a210-8e699dbf01c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:22.228914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa7854-42bb-4383-a210-8e699dbf01c2 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:22.229199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa7854-42bb-4383-a210-8e699dbf01c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:22.229474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa7854-42bb-4383-a210-8e699dbf01c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:22.230581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa7854-42bb-4383-a210-8e699dbf01c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:22.232682 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa7854-42bb-4383-a210-8e699dbf01c2 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:22.237065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 386/777] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:22.240092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa7854-42bb-4383-a210-8e699dbf01c2 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:22.255442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:22.279046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:22.311084 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763bfd6-fc5e-4219-bcb9-e987d76f308c None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:22.347211 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dcf7a-a90b-478e-bfaf-9744dec8cc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:22.347869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dcf7a-a90b-478e-bfaf-9744dec8cc45 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:22.348185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dcf7a-a90b-478e-bfaf-9744dec8cc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:22.348466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dcf7a-a90b-478e-bfaf-9744dec8cc45 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:22.349075 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520dcf7a-a90b-478e-bfaf-9744dec8cc45 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:22.350439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520dcf7a-a90b-478e-bfaf-9744dec8cc45 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:22.351403 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa7854-42bb-4383-a210-8e699dbf01c2 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-520dcf7a-a90b-478e-bfaf-9744dec8cc45 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:22.360772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:22.362136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:22.363256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 392/778] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:28:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:22.368660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dcf7a-a90b-478e-bfaf-9744dec8cc45 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:22.389931 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f5743-fc9f-4c86-b9ec-32578d732ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:22.390821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5743-fc9f-4c86-b9ec-32578d732ee5 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:22.391284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5743-fc9f-4c86-b9ec-32578d732ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:22.391925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5743-fc9f-4c86-b9ec-32578d732ee5 None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:22.392620 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f5743-fc9f-4c86-b9ec-32578d732ee5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:22.394100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f5743-fc9f-4c86-b9ec-32578d732ee5 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:22.436881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 387/779] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:28:22 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:28:22.488765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dcf7a-a90b-478e-bfaf-9744dec8cc45 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:22.526028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ec493-fa65-45bc-9246-7fd877dcfdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:22.526657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ec493-fa65-45bc-9246-7fd877dcfdbf None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:22.526930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ec493-fa65-45bc-9246-7fd877dcfdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:22.527213 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ec493-fa65-45bc-9246-7fd877dcfdbf None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:22.528418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ec493-fa65-45bc-9246-7fd877dcfdbf None admin] RBAC: Adding query filter params (name=compute) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:22.528718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ec493-fa65-45bc-9246-7fd877dcfdbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:22.530133 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ec493-fa65-45bc-9246-7fd877dcfdbf None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:22.532415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:22.541906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 393/780] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:28:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:22.543165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f5743-fc9f-4c86-b9ec-32578d732ee5 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:22.614195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:22.658882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f5743-fc9f-4c86-b9ec-32578d732ee5 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:22.694652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46456252-7a96-42e8-84eb-df62a444389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:22.695258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46456252-7a96-42e8-84eb-df62a444389e None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:22.695533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46456252-7a96-42e8-84eb-df62a444389e None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:22.695800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46456252-7a96-42e8-84eb-df62a444389e None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:22.696425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46456252-7a96-42e8-84eb-df62a444389e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:22.696723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46456252-7a96-42e8-84eb-df62a444389e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:22.698134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46456252-7a96-42e8-84eb-df62a444389e None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:22.703745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 388/781] 10.222.0.17 () {56 vars in 1112 bytes} [Tue Oct 18 11:28:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:22.709527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ec493-fa65-45bc-9246-7fd877dcfdbf None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:22.810831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ec493-fa65-45bc-9246-7fd877dcfdbf None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:22.848136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b823d-0558-465b-a55e-20f54e757f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:22.848763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b823d-0558-465b-a55e-20f54e757f11 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:22.850808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b823d-0558-465b-a55e-20f54e757f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:22.850808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b823d-0558-465b-a55e-20f54e757f11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:22.850808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b823d-0558-465b-a55e-20f54e757f11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:22.851360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b823d-0558-465b-a55e-20f54e757f11 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:22.875246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 394/782] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:28:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:23.493564 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af81d66-2052-4418-8779-01d8a4ef285b None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:23.494065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af81d66-2052-4418-8779-01d8a4ef285b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:23.494065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af81d66-2052-4418-8779-01d8a4ef285b None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:23.495485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 389/783] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:23.501232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71920d-7ec3-41e3-93f7-a9e083758dd2 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:23.501604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71920d-7ec3-41e3-93f7-a9e083758dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:23.501768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71920d-7ec3-41e3-93f7-a9e083758dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:23.520121 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-cb71920d-7ec3-41e3-93f7-a9e083758dd2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:23.521330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cb71920d-7ec3-41e3-93f7-a9e083758dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:23.570563 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:23.596106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 395/784] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:23.874608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af81d66-2052-4418-8779-01d8a4ef285b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:23.978641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af81d66-2052-4418-8779-01d8a4ef285b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:24.004061 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:24.014153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cc0c5-0832-4150-87f9-2098667dbcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:24.014781 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cc0c5-0832-4150-87f9-2098667dbcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:24.015061 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cc0c5-0832-4150-87f9-2098667dbcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:24.015335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599cc0c5-0832-4150-87f9-2098667dbcd3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:24.015914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cc0c5-0832-4150-87f9-2098667dbcd3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:24.017321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599cc0c5-0832-4150-87f9-2098667dbcd3 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-599cc0c5-0832-4150-87f9-2098667dbcd3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:24.026224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:24.027256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:24.033826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 390/785] 10.222.0.17 () {56 vars in 1105 bytes} [Tue Oct 18 11:28:23 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:24.034793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71920d-7ec3-41e3-93f7-a9e083758dd2 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:24.133752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71920d-7ec3-41e3-93f7-a9e083758dd2 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:24.173683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e731191-7e8c-45f9-aa1c-850bab1ed194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:24.174303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e731191-7e8c-45f9-aa1c-850bab1ed194 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:24.174580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e731191-7e8c-45f9-aa1c-850bab1ed194 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:24.174853 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e731191-7e8c-45f9-aa1c-850bab1ed194 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:24.176894 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e731191-7e8c-45f9-aa1c-850bab1ed194 None admin] RBAC: Adding query filter params (name=image) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:24.176894 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e731191-7e8c-45f9-aa1c-850bab1ed194 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:24.177174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e731191-7e8c-45f9-aa1c-850bab1ed194 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:24.187720 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 396/786] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:24.188857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cc0c5-0832-4150-87f9-2098667dbcd3 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:24.289562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599cc0c5-0832-4150-87f9-2098667dbcd3 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:24.322995 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fed5c0-1a48-4efe-9fce-90a15c5dcadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:24.323601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fed5c0-1a48-4efe-9fce-90a15c5dcadb None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:24.323884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fed5c0-1a48-4efe-9fce-90a15c5dcadb None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:24.324207 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fed5c0-1a48-4efe-9fce-90a15c5dcadb None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:24.325420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fed5c0-1a48-4efe-9fce-90a15c5dcadb None admin] RBAC: Adding query filter params (type=image) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:24.325723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fed5c0-1a48-4efe-9fce-90a15c5dcadb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:24.327073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fed5c0-1a48-4efe-9fce-90a15c5dcadb None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:24.336523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 391/787] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:24.337729 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e731191-7e8c-45f9-aa1c-850bab1ed194 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:24.385765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:24.439196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e731191-7e8c-45f9-aa1c-850bab1ed194 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:24.472322 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0983c67-3490-4490-9424-5d50d4a9681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:24.472951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0983c67-3490-4490-9424-5d50d4a9681a None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:24.473248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0983c67-3490-4490-9424-5d50d4a9681a None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:24.473523 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0983c67-3490-4490-9424-5d50d4a9681a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:24.474179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0983c67-3490-4490-9424-5d50d4a9681a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:24.475516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0983c67-3490-4490-9424-5d50d4a9681a None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:24.496722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 397/788] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:28:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:28:24.660172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bdc0a-f070-4c58-9434-29d7868c982d None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:24.660672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bdc0a-f070-4c58-9434-29d7868c982d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:24.660672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bdc0a-f070-4c58-9434-29d7868c982d None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:24.662129 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 392/789] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:24.667449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d4c27-4d20-43ef-b6c0-c008ebcc80e5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:24.667892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d4c27-4d20-43ef-b6c0-c008ebcc80e5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:24.668606 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d4c27-4d20-43ef-b6c0-c008ebcc80e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:24.687428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-be1d4c27-4d20-43ef-b6c0-c008ebcc80e5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:24.688619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-be1d4c27-4d20-43ef-b6c0-c008ebcc80e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:24.766016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 398/790] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:25.037251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bdc0a-f070-4c58-9434-29d7868c982d None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:25.093385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:25.109563 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277712a7-49f3-4cf3-808d-adce81352b07 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:25.109879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277712a7-49f3-4cf3-808d-adce81352b07 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:25.110116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277712a7-49f3-4cf3-808d-adce81352b07 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:25.111593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 399/791] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:25.116605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0947130-63e6-4ff0-9297-07a52ee4824a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:25.116916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0947130-63e6-4ff0-9297-07a52ee4824a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:25.117195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0947130-63e6-4ff0-9297-07a52ee4824a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:25.134863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d0947130-63e6-4ff0-9297-07a52ee4824a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:25.136060 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d0947130-63e6-4ff0-9297-07a52ee4824a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:25.142341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bdc0a-f070-4c58-9434-29d7868c982d None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:25.159287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:25.179992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8055c49-dd61-473c-b125-5d49ec65268a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:25.179992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8055c49-dd61-473c-b125-5d49ec65268a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:25.179992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8055c49-dd61-473c-b125-5d49ec65268a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:25.179992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8055c49-dd61-473c-b125-5d49ec65268a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:25.180626 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8055c49-dd61-473c-b125-5d49ec65268a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:25.182063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8055c49-dd61-473c-b125-5d49ec65268a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:25.183792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 393/792] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:25.189277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8055c49-dd61-473c-b125-5d49ec65268a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:25.221025 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 400/793] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:28:25.294529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8055c49-dd61-473c-b125-5d49ec65268a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:25.316051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:25.329046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f1d10-0177-4337-aa2c-bcbaa55df7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:25.329651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f1d10-0177-4337-aa2c-bcbaa55df7e7 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:25.329927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f1d10-0177-4337-aa2c-bcbaa55df7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:25.330194 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f1d10-0177-4337-aa2c-bcbaa55df7e7 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:25.330837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f1d10-0177-4337-aa2c-bcbaa55df7e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:25.332354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f1d10-0177-4337-aa2c-bcbaa55df7e7 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:25.360879 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 394/794] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:28:25 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:28:25.498950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0947130-63e6-4ff0-9297-07a52ee4824a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:25.604193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0947130-63e6-4ff0-9297-07a52ee4824a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:25.640594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e45ed8-aa49-4602-9e32-345c9f0e02b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:25.641306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e45ed8-aa49-4602-9e32-345c9f0e02b4 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:25.641590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e45ed8-aa49-4602-9e32-345c9f0e02b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:25.641870 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e45ed8-aa49-4602-9e32-345c9f0e02b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:25.653814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e45ed8-aa49-4602-9e32-345c9f0e02b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-f0e45ed8-aa49-4602-9e32-345c9f0e02b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:28:25.658609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:25.659930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:25.661232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 401/795] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Oct 18 11:28:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:25.665830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f1d10-0177-4337-aa2c-bcbaa55df7e7 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:25.769847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f1d10-0177-4337-aa2c-bcbaa55df7e7 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:25.808579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b5b35-2dca-498a-9ab5-70f8948cd3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:25.809744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b5b35-2dca-498a-9ab5-70f8948cd3ca None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:25.810022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b5b35-2dca-498a-9ab5-70f8948cd3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:25.810292 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b5b35-2dca-498a-9ab5-70f8948cd3ca None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:25.810963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b5b35-2dca-498a-9ab5-70f8948cd3ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:25.811268 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b5b35-2dca-498a-9ab5-70f8948cd3ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:25.812642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b5b35-2dca-498a-9ab5-70f8948cd3ca None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:25.821461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 395/796] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Oct 18 11:28:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:25.828347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e45ed8-aa49-4602-9e32-345c9f0e02b4 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:25.848019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:25.941564 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e45ed8-aa49-4602-9e32-345c9f0e02b4 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:25.975917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b573ecb-bd89-44a5-bff4-c3e093294654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:25.976779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b573ecb-bd89-44a5-bff4-c3e093294654 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:25.977247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b573ecb-bd89-44a5-bff4-c3e093294654 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:25.977674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b573ecb-bd89-44a5-bff4-c3e093294654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:25.987598 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b573ecb-bd89-44a5-bff4-c3e093294654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:25.989544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b573ecb-bd89-44a5-bff4-c3e093294654 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-1b573ecb-bd89-44a5-bff4-c3e093294654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:28:26.005253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:26.006242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:28:26.007217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:28:26.007217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:26.007217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:26.007217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 402/797] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:26.014101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b5b35-2dca-498a-9ab5-70f8948cd3ca None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:26.078934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:26.141199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b5b35-2dca-498a-9ab5-70f8948cd3ca None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:26.181886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86748244-3c51-4e50-b443-b45729e2380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:26.182505 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86748244-3c51-4e50-b443-b45729e2380c None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:26.182781 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86748244-3c51-4e50-b443-b45729e2380c None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:26.183050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86748244-3c51-4e50-b443-b45729e2380c None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:26.183734 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86748244-3c51-4e50-b443-b45729e2380c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:26.184541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86748244-3c51-4e50-b443-b45729e2380c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:26.185941 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86748244-3c51-4e50-b443-b45729e2380c None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:26.203138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 396/798] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:26.204684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b573ecb-bd89-44a5-bff4-c3e093294654 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:26.299987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:26.333108 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b573ecb-bd89-44a5-bff4-c3e093294654 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:26.367658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd89e30-de87-4d24-a14c-c826ec40afc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:26.368285 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd89e30-de87-4d24-a14c-c826ec40afc0 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:26.368555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd89e30-de87-4d24-a14c-c826ec40afc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:26.368822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd89e30-de87-4d24-a14c-c826ec40afc0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:26.376959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd89e30-de87-4d24-a14c-c826ec40afc0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:26.378373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd89e30-de87-4d24-a14c-c826ec40afc0 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-bcd89e30-de87-4d24-a14c-c826ec40afc0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:28:26.389301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:26.390224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:26.391187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:26.391187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:26.391187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:28:26.391187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:28:26.391187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:28:26.391187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:28:26.391187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:28:26.391187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:28:26.391187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:28:26.391187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:26.392909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 403/799] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Oct 18 11:28:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:26.396346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86748244-3c51-4e50-b443-b45729e2380c None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:26.514086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:26.518017 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86748244-3c51-4e50-b443-b45729e2380c None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:26.552924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555557be-2b37-4ccc-a4c4-9605262c3533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:26.553526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555557be-2b37-4ccc-a4c4-9605262c3533 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:26.553799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555557be-2b37-4ccc-a4c4-9605262c3533 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:26.554060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555557be-2b37-4ccc-a4c4-9605262c3533 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:26.554777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555557be-2b37-4ccc-a4c4-9605262c3533 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:26.555080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555557be-2b37-4ccc-a4c4-9605262c3533 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:26.556485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555557be-2b37-4ccc-a4c4-9605262c3533 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:26.577767 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 397/800] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Oct 18 11:28:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:26.578981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd89e30-de87-4d24-a14c-c826ec40afc0 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:26.701713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd89e30-de87-4d24-a14c-c826ec40afc0 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:26.726779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66c995-d230-401b-b2a6-fed65f2cde73 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:26.727280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66c995-d230-401b-b2a6-fed65f2cde73 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:26.727380 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66c995-d230-401b-b2a6-fed65f2cde73 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:26.728641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 398/801] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:26.731613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:26.734156 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5660a-85df-4cfc-9571-bb35d26ef542 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:26.734379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5660a-85df-4cfc-9571-bb35d26ef542 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:26.734655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5660a-85df-4cfc-9571-bb35d26ef542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:26.739551 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65d6c9-f723-4f29-b560-ce336ccda199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:26.740249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d6c9-f723-4f29-b560-ce336ccda199 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:26.740525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d6c9-f723-4f29-b560-ce336ccda199 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:26.740792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d6c9-f723-4f29-b560-ce336ccda199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:26.751898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65d6c9-f723-4f29-b560-ce336ccda199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:26.753063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-23d5660a-85df-4cfc-9571-bb35d26ef542 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:26.753374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65d6c9-f723-4f29-b560-ce336ccda199 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:26.754365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-23d5660a-85df-4cfc-9571-bb35d26ef542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-fa65d6c9-f723-4f29-b560-ce336ccda199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:28:26.768030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:26.768995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:28:26.769882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:28:26.769882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:26.769882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:26.769882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 404/802] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:26.774759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65d6c9-f723-4f29-b560-ce336ccda199 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:26.831291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 399/803] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:28:26.882290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65d6c9-f723-4f29-b560-ce336ccda199 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:26.917244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5383fe-51e5-43ee-9d41-e85eda04007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:26.917855 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5383fe-51e5-43ee-9d41-e85eda04007d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:26.918161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5383fe-51e5-43ee-9d41-e85eda04007d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:26.918436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5383fe-51e5-43ee-9d41-e85eda04007d None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:26.919074 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5383fe-51e5-43ee-9d41-e85eda04007d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:26.919379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5383fe-51e5-43ee-9d41-e85eda04007d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:26.920739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5383fe-51e5-43ee-9d41-e85eda04007d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:26.936973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 405/804] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:26.938730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5660a-85df-4cfc-9571-bb35d26ef542 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:27.032406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:27.048366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5660a-85df-4cfc-9571-bb35d26ef542 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:27.090244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f674bf-b366-413a-b50d-4ab1137a92d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:27.091020 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f674bf-b366-413a-b50d-4ab1137a92d3 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:27.091282 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f674bf-b366-413a-b50d-4ab1137a92d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:27.091595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f674bf-b366-413a-b50d-4ab1137a92d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:27.098086 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5383fe-51e5-43ee-9d41-e85eda04007d None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:27.099370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f674bf-b366-413a-b50d-4ab1137a92d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:27.101018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f674bf-b366-413a-b50d-4ab1137a92d3 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-52f674bf-b366-413a-b50d-4ab1137a92d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:28:27.115592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:27.116495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:27.117426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:28:27.117426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:28:27.117426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:28:27.117426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:28:27.117426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:28:27.117426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:28:27.117426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:27.117426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:27.117426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 400/805] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:28:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:27.123256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f674bf-b366-413a-b50d-4ab1137a92d3 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:27.145872 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:27.210697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5383fe-51e5-43ee-9d41-e85eda04007d None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:27.240797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f674bf-b366-413a-b50d-4ab1137a92d3 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:27.263037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a2107-7ab4-4ae8-b354-14ef8e91436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:27.263802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a2107-7ab4-4ae8-b354-14ef8e91436d None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:27.264175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a2107-7ab4-4ae8-b354-14ef8e91436d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:27.264500 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a2107-7ab4-4ae8-b354-14ef8e91436d None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:27.265324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a2107-7ab4-4ae8-b354-14ef8e91436d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:27.267031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a2107-7ab4-4ae8-b354-14ef8e91436d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:27.278010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.sql.core [None req-064a2107-7ab4-4ae8-b354-14ef8e91436d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 18 11:28:27.278010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 18 11:28:27.278010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [parameters: {'id': '1b7417f141d2467085cd65ff13985487', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 18 11:28:27.278010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81701) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-064a2107-7ab4-4ae8-b354-14ef8e91436d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result = self._query(query) Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application conn.query(q) Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 18 11:28:27.288065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result.read() Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 18 11:28:27.288980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application next(self.gen) Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application yield resource Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application next(self.gen) Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.session.rollback() Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.force_reraise() Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise self.value Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application session.commit() Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 18 11:28:27.290071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._prepare_impl() Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.session.flush() Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._flush(objects) Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application compat.raise_( Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application flush_context.execute() Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rec.execute(self) Oct 18 11:28:27.290952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 18 11:28:27.291842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result = self._query(query) Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application conn.query(q) Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application result.read() Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 18 11:28:27.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [parameters: {'id': '1b7417f141d2467085cd65ff13985487', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 18 11:28:27.294344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:27.295666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:27.297392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 406/806] 10.222.0.17 () {60 vars in 1137 bytes} [Tue Oct 18 11:28:27 2022] POST /identity/v3/domains => generated 190 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:27.298445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d2b2e-1b4b-4ec9-b878-8ad98cb4affc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:27.298445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d2b2e-1b4b-4ec9-b878-8ad98cb4affc None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:27.298445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d2b2e-1b4b-4ec9-b878-8ad98cb4affc None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:27.298445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d2b2e-1b4b-4ec9-b878-8ad98cb4affc None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:27.299216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d2b2e-1b4b-4ec9-b878-8ad98cb4affc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:27.299643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d2b2e-1b4b-4ec9-b878-8ad98cb4affc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:27.307653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d2b2e-1b4b-4ec9-b878-8ad98cb4affc None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:27.307821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a2107-7ab4-4ae8-b354-14ef8e91436d None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:27.319190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 401/807] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:28:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:27.324414 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d2b2e-1b4b-4ec9-b878-8ad98cb4affc None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:27.334404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:27.411572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a2107-7ab4-4ae8-b354-14ef8e91436d None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:27.420683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:27.439481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d2b2e-1b4b-4ec9-b878-8ad98cb4affc None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:27.452818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08828b6-5e39-4f32-856e-79f82c90518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:27.453515 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08828b6-5e39-4f32-856e-79f82c90518b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:27.453793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08828b6-5e39-4f32-856e-79f82c90518b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:27.454060 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08828b6-5e39-4f32-856e-79f82c90518b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:27.466768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08828b6-5e39-4f32-856e-79f82c90518b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:27.468279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08828b6-5e39-4f32-856e-79f82c90518b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:27.489323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffec06-875a-4040-9a29-566dc2be3afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:27.490006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffec06-875a-4040-9a29-566dc2be3afd None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:27.490279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffec06-875a-4040-9a29-566dc2be3afd None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:27.490553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffec06-875a-4040-9a29-566dc2be3afd None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/87f8a4a8277745a9ad1976af9df31cb9/roles/67d4f6145bce41c6a2168196bb6d8a74` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-f08828b6-5e39-4f32-856e-79f82c90518b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:28:27.492649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:27.493937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:28:27.495366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:28:27.495366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:27.495366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:27.495639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffec06-875a-4040-9a29-566dc2be3afd None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=87f8a4a8277745a9ad1976af9df31cb9, role_id=67d4f6145bce41c6a2168196bb6d8a74)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:27.495639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffec06-875a-4040-9a29-566dc2be3afd None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:27.497383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 407/808] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:27.503405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08828b6-5e39-4f32-856e-79f82c90518b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:27.512492 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 402/809] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:28:27 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/87f8a4a8277745a9ad1976af9df31cb9/roles/67d4f6145bce41c6a2168196bb6d8a74 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:28:27.581151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375f89-7535-40a6-8106-56bb71e035d1 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:27.581358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375f89-7535-40a6-8106-56bb71e035d1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:27.581593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375f89-7535-40a6-8106-56bb71e035d1 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:27.585448 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 403/810] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:27.588692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb51d33-95b2-4772-84f9-4e739485e9c7 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:27.589216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb51d33-95b2-4772-84f9-4e739485e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:27.589488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb51d33-95b2-4772-84f9-4e739485e9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:27.609404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-bbb51d33-95b2-4772-84f9-4e739485e9c7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:27.610638 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-bbb51d33-95b2-4772-84f9-4e739485e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:27.627655 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08828b6-5e39-4f32-856e-79f82c90518b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:27.673992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf7e98-4ee1-4b86-b1ec-351792a03e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:27.674939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf7e98-4ee1-4b86-b1ec-351792a03e1f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:27.675432 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf7e98-4ee1-4b86-b1ec-351792a03e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:27.675869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf7e98-4ee1-4b86-b1ec-351792a03e1f None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:27.676821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf7e98-4ee1-4b86-b1ec-351792a03e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:27.677167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf7e98-4ee1-4b86-b1ec-351792a03e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:27.678596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf7e98-4ee1-4b86-b1ec-351792a03e1f None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:27.692884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 408/811] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:27.699005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:27.706715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 404/812] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:27.976348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf7e98-4ee1-4b86-b1ec-351792a03e1f None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:27.996806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:28.090008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf7e98-4ee1-4b86-b1ec-351792a03e1f None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:28.125045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64533c2b-bb29-486c-b33b-bd38fba0a2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:28.125673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64533c2b-bb29-486c-b33b-bd38fba0a2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:28.125948 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64533c2b-bb29-486c-b33b-bd38fba0a2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:28.126219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64533c2b-bb29-486c-b33b-bd38fba0a2f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:28.135999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64533c2b-bb29-486c-b33b-bd38fba0a2f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-64533c2b-bb29-486c-b33b-bd38fba0a2f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:28:28.141376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:28.143038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:28.144444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 409/813] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:28:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:28.148530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb51d33-95b2-4772-84f9-4e739485e9c7 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:28.257025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb51d33-95b2-4772-84f9-4e739485e9c7 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:28.295197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdc3be-800d-43a4-bd02-53a88261fa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:28.295839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdc3be-800d-43a4-bd02-53a88261fa3b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:28.296158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdc3be-800d-43a4-bd02-53a88261fa3b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:28.296475 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdc3be-800d-43a4-bd02-53a88261fa3b None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:28.297230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdc3be-800d-43a4-bd02-53a88261fa3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:28.297583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdc3be-800d-43a4-bd02-53a88261fa3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:28.299115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdc3be-800d-43a4-bd02-53a88261fa3b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:28.308358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 405/814] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:28:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:28.316357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64533c2b-bb29-486c-b33b-bd38fba0a2f2 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:28.393380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:28.422300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64533c2b-bb29-486c-b33b-bd38fba0a2f2 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:28.466545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64a97a-213c-4b05-bfbe-9a79bcae0785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:28.467342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64a97a-213c-4b05-bfbe-9a79bcae0785 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:28.467683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64a97a-213c-4b05-bfbe-9a79bcae0785 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:28.468039 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64a97a-213c-4b05-bfbe-9a79bcae0785 None admin] PATH_INFO: `/v3/users/672302d3168e452a96e656b205c1f5ca` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:28.484323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64a97a-213c-4b05-bfbe-9a79bcae0785 None admin] RBAC: Authorizing `identity:get_user(user_id=672302d3168e452a96e656b205c1f5ca)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:28.485898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64a97a-213c-4b05-bfbe-9a79bcae0785 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:28.487466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 410/815] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:28:28 2022] GET /identity/v3/users/672302d3168e452a96e656b205c1f5ca => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:28.493552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdc3be-800d-43a4-bd02-53a88261fa3b None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:28.551621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:28.591345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdc3be-800d-43a4-bd02-53a88261fa3b None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:28.629747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a80d4-5dbc-425e-bee1-5d3cc59d6b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:28.630452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a80d4-5dbc-425e-bee1-5d3cc59d6b75 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:28.630789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a80d4-5dbc-425e-bee1-5d3cc59d6b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:28.631089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a80d4-5dbc-425e-bee1-5d3cc59d6b75 None admin] PATH_INFO: `/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:28.632364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a80d4-5dbc-425e-bee1-5d3cc59d6b75 None admin] RBAC: Authorizing `identity:get_project(project_id=cd9ac69fc9d44964b76ec47120c06b9f)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:28.634060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a80d4-5dbc-425e-bee1-5d3cc59d6b75 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:28.635919 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 406/816] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:28:28 2022] GET /identity/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:28.643213 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64a97a-213c-4b05-bfbe-9a79bcae0785 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:28.745458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64a97a-213c-4b05-bfbe-9a79bcae0785 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:28.778927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd4785-bb10-4e76-bae0-86771ebfb220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:28.779584 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd4785-bb10-4e76-bae0-86771ebfb220 None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:28.779869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd4785-bb10-4e76-bae0-86771ebfb220 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:28.780141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd4785-bb10-4e76-bae0-86771ebfb220 None admin] PATH_INFO: `/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f/users/672302d3168e452a96e656b205c1f5ca/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:28.782058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd4785-bb10-4e76-bae0-86771ebfb220 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd9ac69fc9d44964b76ec47120c06b9f, user_id=672302d3168e452a96e656b205c1f5ca, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:28.783495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd4785-bb10-4e76-bae0-86771ebfb220 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:28.786592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:28.798198 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 411/817] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:28:28 2022] PUT /identity/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f/users/672302d3168e452a96e656b205c1f5ca/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:28:29.679178 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146e2f3-af8f-4d90-8d0d-b4220d78963c None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:29.679731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146e2f3-af8f-4d90-8d0d-b4220d78963c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:29.679843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146e2f3-af8f-4d90-8d0d-b4220d78963c None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:29.681299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 407/818] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:29.687294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927b1b8-af61-4a1b-bab6-a3a70cfa9e82 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:29.687676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927b1b8-af61-4a1b-bab6-a3a70cfa9e82 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:29.687941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927b1b8-af61-4a1b-bab6-a3a70cfa9e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:29.706614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e927b1b8-af61-4a1b-bab6-a3a70cfa9e82 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:29.707797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e927b1b8-af61-4a1b-bab6-a3a70cfa9e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:29.774498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 412/819] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:28:30.042719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146e2f3-af8f-4d90-8d0d-b4220d78963c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:30.144294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146e2f3-af8f-4d90-8d0d-b4220d78963c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:30.170361 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:30.180944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a09c5e-9c9c-41af-a80b-430747a7227d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:30.181580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a09c5e-9c9c-41af-a80b-430747a7227d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:30.181854 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a09c5e-9c9c-41af-a80b-430747a7227d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:30.182119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a09c5e-9c9c-41af-a80b-430747a7227d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:30.193374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a09c5e-9c9c-41af-a80b-430747a7227d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-08a09c5e-9c9c-41af-a80b-430747a7227d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:28:30.198135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:30.199745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:30.200615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 408/820] 10.222.0.17 () {56 vars in 1120 bytes} [Tue Oct 18 11:28:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:30.205902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927b1b8-af61-4a1b-bab6-a3a70cfa9e82 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:30.300916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927b1b8-af61-4a1b-bab6-a3a70cfa9e82 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:30.326427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:30.335437 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2d63e-c828-4c1c-ab11-ebbe77a6713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:30.336049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2d63e-c828-4c1c-ab11-ebbe77a6713b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:30.336328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2d63e-c828-4c1c-ab11-ebbe77a6713b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:30.336596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2d63e-c828-4c1c-ab11-ebbe77a6713b None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:30.337290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2d63e-c828-4c1c-ab11-ebbe77a6713b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:30.337592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2d63e-c828-4c1c-ab11-ebbe77a6713b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:30.338947 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2d63e-c828-4c1c-ab11-ebbe77a6713b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:30.347232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 413/821] 10.222.0.17 () {56 vars in 1115 bytes} [Tue Oct 18 11:28:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:30.353133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a09c5e-9c9c-41af-a80b-430747a7227d None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:30.449717 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a09c5e-9c9c-41af-a80b-430747a7227d None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:30.481754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da012b6-e181-43a4-8ecc-c682800eebce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:30.482383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da012b6-e181-43a4-8ecc-c682800eebce None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:30.482661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da012b6-e181-43a4-8ecc-c682800eebce None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:30.482925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da012b6-e181-43a4-8ecc-c682800eebce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:30.493433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da012b6-e181-43a4-8ecc-c682800eebce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:30.494871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da012b6-e181-43a4-8ecc-c682800eebce None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-2da012b6-e181-43a4-8ecc-c682800eebce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:28:30.509047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:30.510143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:28:30.511042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:28:30.511042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:30.511042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:30.511042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 409/822] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:30.516493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2d63e-c828-4c1c-ab11-ebbe77a6713b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:30.613965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2d63e-c828-4c1c-ab11-ebbe77a6713b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:30.646521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4169f9-1da0-4a38-b800-17a1b7b39373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:30.647123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4169f9-1da0-4a38-b800-17a1b7b39373 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:30.647412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4169f9-1da0-4a38-b800-17a1b7b39373 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:30.647671 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4169f9-1da0-4a38-b800-17a1b7b39373 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:30.648304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4169f9-1da0-4a38-b800-17a1b7b39373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:30.648604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4169f9-1da0-4a38-b800-17a1b7b39373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:30.649998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4169f9-1da0-4a38-b800-17a1b7b39373 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:30.661277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 414/823] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:30.666874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da012b6-e181-43a4-8ecc-c682800eebce None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:30.708719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:30.765251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da012b6-e181-43a4-8ecc-c682800eebce None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:30.797997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8dc9d-5ceb-4392-91fd-bada64daa410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:30.798625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8dc9d-5ceb-4392-91fd-bada64daa410 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:30.798903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8dc9d-5ceb-4392-91fd-bada64daa410 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:30.799174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8dc9d-5ceb-4392-91fd-bada64daa410 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:30.806524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8dc9d-5ceb-4392-91fd-bada64daa410 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:30.807944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8dc9d-5ceb-4392-91fd-bada64daa410 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-8be8dc9d-5ceb-4392-91fd-bada64daa410 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:28:30.819364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:30.820294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:30.821311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 410/824] 10.222.0.17 () {56 vars in 1093 bytes} [Tue Oct 18 11:28:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:30.826255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4169f9-1da0-4a38-b800-17a1b7b39373 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:30.879245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:30.931624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4169f9-1da0-4a38-b800-17a1b7b39373 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:30.956462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a4931-f2db-4f59-9b6b-a6f448a8ba8f None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:30.956743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a4931-f2db-4f59-9b6b-a6f448a8ba8f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:30.957053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a4931-f2db-4f59-9b6b-a6f448a8ba8f None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:30.958376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 411/825] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:30.963719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e5377-9dbd-4d32-b5cc-a25a8987068d None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:30.964002 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e5377-9dbd-4d32-b5cc-a25a8987068d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:30.964273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e5377-9dbd-4d32-b5cc-a25a8987068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:30.971642 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba33b7-b110-440f-863c-0455b9d9fc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:30.972252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba33b7-b110-440f-863c-0455b9d9fc96 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:30.972527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba33b7-b110-440f-863c-0455b9d9fc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:30.972792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba33b7-b110-440f-863c-0455b9d9fc96 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:30.973556 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba33b7-b110-440f-863c-0455b9d9fc96 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:30.973856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba33b7-b110-440f-863c-0455b9d9fc96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:30.975277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba33b7-b110-440f-863c-0455b9d9fc96 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:30.985122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-b81e5377-9dbd-4d32-b5cc-a25a8987068d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:30.986327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b81e5377-9dbd-4d32-b5cc-a25a8987068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:30.993873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 415/826] 10.222.0.17 () {56 vars in 1133 bytes} [Tue Oct 18 11:28:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:30.999583 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba33b7-b110-440f-863c-0455b9d9fc96 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:31.054225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 412/827] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:28:31.102881 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba33b7-b110-440f-863c-0455b9d9fc96 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:31.110981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:31.139765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bddea-fdf8-4831-a632-d7b6c931c542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:31.140288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bddea-fdf8-4831-a632-d7b6c931c542 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:31.140558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bddea-fdf8-4831-a632-d7b6c931c542 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:31.140823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bddea-fdf8-4831-a632-d7b6c931c542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:31.210599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bddea-fdf8-4831-a632-d7b6c931c542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:31.212044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bddea-fdf8-4831-a632-d7b6c931c542 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:31.309529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 416/828] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:28:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 18 11:28:31.315116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e5377-9dbd-4d32-b5cc-a25a8987068d None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:31.317474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bddea-fdf8-4831-a632-d7b6c931c542 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:31.326389 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:31.346866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:31.416391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e5377-9dbd-4d32-b5cc-a25a8987068d None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:31.416732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bddea-fdf8-4831-a632-d7b6c931c542 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:31.448561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b77c7-62c2-48f0-92cd-e0399ea07bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:31.449747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b77c7-62c2-48f0-92cd-e0399ea07bed None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:31.450029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b77c7-62c2-48f0-92cd-e0399ea07bed None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:31.450181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3d0b-6dda-4c39-958d-a55b208527ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:31.450287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b77c7-62c2-48f0-92cd-e0399ea07bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:31.450803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3d0b-6dda-4c39-958d-a55b208527ef None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:31.451077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3d0b-6dda-4c39-958d-a55b208527ef None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:31.451343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3d0b-6dda-4c39-958d-a55b208527ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:31.460100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b77c7-62c2-48f0-92cd-e0399ea07bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:31.461434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df3d0b-6dda-4c39-958d-a55b208527ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:28:31.461561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b77c7-62c2-48f0-92cd-e0399ea07bed None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-42df3d0b-6dda-4c39-958d-a55b208527ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:28:31.466753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:31.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:31.468651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 417/829] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:31.473803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3d0b-6dda-4c39-958d-a55b208527ef None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-6c9b77c7-62c2-48f0-92cd-e0399ea07bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:28:31.477625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:31.478702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:28:31.479610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:28:31.479610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:28:31.479610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:31.479610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 413/830] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:31.484318 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b77c7-62c2-48f0-92cd-e0399ea07bed None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:31.537670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:31.564696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:31.572511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3d0b-6dda-4c39-958d-a55b208527ef None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:31.583351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b77c7-62c2-48f0-92cd-e0399ea07bed None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:31.606288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c4dad-5a6f-4ed5-a6ee-ca7b4b129b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:31.606904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c4dad-5a6f-4ed5-a6ee-ca7b4b129b29 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:31.607183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c4dad-5a6f-4ed5-a6ee-ca7b4b129b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:31.607499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c4dad-5a6f-4ed5-a6ee-ca7b4b129b29 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:31.608140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c4dad-5a6f-4ed5-a6ee-ca7b4b129b29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:31.608437 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c4dad-5a6f-4ed5-a6ee-ca7b4b129b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:31.609819 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c4dad-5a6f-4ed5-a6ee-ca7b4b129b29 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:31.616203 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c18c18-4bf6-4811-a6f3-8dc9a07d1aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:31.616797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c18c18-4bf6-4811-a6f3-8dc9a07d1aeb None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:31.617100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c18c18-4bf6-4811-a6f3-8dc9a07d1aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:31.617367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c18c18-4bf6-4811-a6f3-8dc9a07d1aeb None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:31.617943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 418/831] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:31.618065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c18c18-4bf6-4811-a6f3-8dc9a07d1aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:31.618281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c18c18-4bf6-4811-a6f3-8dc9a07d1aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:31.619691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c18c18-4bf6-4811-a6f3-8dc9a07d1aeb None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:31.623762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c4dad-5a6f-4ed5-a6ee-ca7b4b129b29 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:31.631566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 414/832] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:31.637690 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c18c18-4bf6-4811-a6f3-8dc9a07d1aeb None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:31.726202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c4dad-5a6f-4ed5-a6ee-ca7b4b129b29 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:31.734368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c18c18-4bf6-4811-a6f3-8dc9a07d1aeb None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:31.751933 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:31.760528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c777fb-952d-43fd-8b1d-534f70c3e9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:31.761178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c777fb-952d-43fd-8b1d-534f70c3e9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:31.761455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c777fb-952d-43fd-8b1d-534f70c3e9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:31.762267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c777fb-952d-43fd-8b1d-534f70c3e9b1 None admin] PATH_INFO: `/v3/users/672302d3168e452a96e656b205c1f5ca` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:31.763650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c777fb-952d-43fd-8b1d-534f70c3e9b1 None admin] RBAC: Authorizing `identity:get_user(user_id=672302d3168e452a96e656b205c1f5ca)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:31.765103 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c777fb-952d-43fd-8b1d-534f70c3e9b1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:31.766692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 419/833] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:28:31 2022] GET /identity/v3/users/672302d3168e452a96e656b205c1f5ca => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:31.768915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a0a56-b44c-4de9-a112-170afd7c83f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:31.769557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a0a56-b44c-4de9-a112-170afd7c83f0 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:31.769833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a0a56-b44c-4de9-a112-170afd7c83f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:31.770100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a0a56-b44c-4de9-a112-170afd7c83f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:31.772517 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c777fb-952d-43fd-8b1d-534f70c3e9b1 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:31.777359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a0a56-b44c-4de9-a112-170afd7c83f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:31.778803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a0a56-b44c-4de9-a112-170afd7c83f0 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-818a0a56-b44c-4de9-a112-170afd7c83f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:28:31.791085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:31.792234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:31.793395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:28:31.793395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:28:31.793395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:28:31.793395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:28:31.793395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:28:31.793395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:28:31.793395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:28:31.793395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:31.793395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 415/834] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:28:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:28:31.798185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a0a56-b44c-4de9-a112-170afd7c83f0 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:31.801510 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:31.880347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c777fb-952d-43fd-8b1d-534f70c3e9b1 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:31.906453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a0a56-b44c-4de9-a112-170afd7c83f0 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:31.914299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435793ab-cfa3-4f5d-a5c8-9656cf144d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:31.914910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435793ab-cfa3-4f5d-a5c8-9656cf144d57 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:31.915186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435793ab-cfa3-4f5d-a5c8-9656cf144d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:31.915458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435793ab-cfa3-4f5d-a5c8-9656cf144d57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:31.940558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764de47-9b1a-4395-88b1-dafca5791637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:31.941196 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764de47-9b1a-4395-88b1-dafca5791637 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:31.941473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764de47-9b1a-4395-88b1-dafca5791637 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:31.941752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764de47-9b1a-4395-88b1-dafca5791637 None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:31.942435 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764de47-9b1a-4395-88b1-dafca5791637 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:31.942738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764de47-9b1a-4395-88b1-dafca5791637 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:31.944170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764de47-9b1a-4395-88b1-dafca5791637 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:31.956900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 416/835] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:28:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:31.962847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764de47-9b1a-4395-88b1-dafca5791637 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:31.981147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:31.988918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435793ab-cfa3-4f5d-a5c8-9656cf144d57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:31.990374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435793ab-cfa3-4f5d-a5c8-9656cf144d57 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:32.027113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:32.066863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764de47-9b1a-4395-88b1-dafca5791637 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:32.088861 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 420/836] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:28:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 18 11:28:32.094630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435793ab-cfa3-4f5d-a5c8-9656cf144d57 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:32.100106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c6a2c-583b-43da-ac02-aeecc12b0a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:32.100710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c6a2c-583b-43da-ac02-aeecc12b0a21 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:32.101022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c6a2c-583b-43da-ac02-aeecc12b0a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:32.101296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c6a2c-583b-43da-ac02-aeecc12b0a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:32.102017 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c6a2c-583b-43da-ac02-aeecc12b0a21 None admin] RBAC: Adding query filter params (role.id=67d4f6145bce41c6a2168196bb6d8a74, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=87f8a4a8277745a9ad1976af9df31cb9) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:32.102323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c6a2c-583b-43da-ac02-aeecc12b0a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:32.103749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c6a2c-583b-43da-ac02-aeecc12b0a21 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:32.110262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 417/837] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:28:31 2022] GET /identity/v3/role_assignments?user.id=87f8a4a8277745a9ad1976af9df31cb9&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=67d4f6145bce41c6a2168196bb6d8a74 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:32.198293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:32.205019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435793ab-cfa3-4f5d-a5c8-9656cf144d57 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:32.243862 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc6d72-245a-461f-9468-cf5d03fc5595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:32.244436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc6d72-245a-461f-9468-cf5d03fc5595 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:32.244711 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc6d72-245a-461f-9468-cf5d03fc5595 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:32.245022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc6d72-245a-461f-9468-cf5d03fc5595 None admin] PATH_INFO: `/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:32.246935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc6d72-245a-461f-9468-cf5d03fc5595 None admin] RBAC: Authorizing `identity:get_project(project_id=cd9ac69fc9d44964b76ec47120c06b9f)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:32.248415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc6d72-245a-461f-9468-cf5d03fc5595 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:32.250183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 421/838] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:28:32 2022] GET /identity/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:32.256578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c6a2c-583b-43da-ac02-aeecc12b0a21 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:32.366040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c6a2c-583b-43da-ac02-aeecc12b0a21 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:32.405445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:32.415508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaf871-bb7d-4d27-9608-e411637df1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:32.416135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaf871-bb7d-4d27-9608-e411637df1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:32.416412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaf871-bb7d-4d27-9608-e411637df1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:32.416677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaf871-bb7d-4d27-9608-e411637df1f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:32.417525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaf871-bb7d-4d27-9608-e411637df1f5 None admin] RBAC: Adding query filter params (role.id=8d9ff9da6ad946a6bc49336777182ff9, scope.project.id=cd9ac69fc9d44964b76ec47120c06b9f, user.id=672302d3168e452a96e656b205c1f5ca) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:32.418368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaf871-bb7d-4d27-9608-e411637df1f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:32.419818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaf871-bb7d-4d27-9608-e411637df1f5 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:32.426888 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 418/839] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:28:32 2022] GET /identity/v3/role_assignments?user.id=672302d3168e452a96e656b205c1f5ca&scope.project.id=cd9ac69fc9d44964b76ec47120c06b9f&role.id=8d9ff9da6ad946a6bc49336777182ff9 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:34.579700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce492-fdef-4124-ba00-a9373b9dae63 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:34.580218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce492-fdef-4124-ba00-a9373b9dae63 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:34.580218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce492-fdef-4124-ba00-a9373b9dae63 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:34.581487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 422/840] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:34.586568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa3dbc-60b0-4728-93fa-192bfa13f049 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:34.586974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa3dbc-60b0-4728-93fa-192bfa13f049 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:34.587114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa3dbc-60b0-4728-93fa-192bfa13f049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:34.605416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6bfa3dbc-60b0-4728-93fa-192bfa13f049 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:34.606611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6bfa3dbc-60b0-4728-93fa-192bfa13f049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:34.670429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 419/841] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:34.938903 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce492-fdef-4124-ba00-a9373b9dae63 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:35.038964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce492-fdef-4124-ba00-a9373b9dae63 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:35.073125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a33ec-fce1-4e25-ad9a-5738f1c9f16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:35.073760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a33ec-fce1-4e25-ad9a-5738f1c9f16a None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:35.074036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a33ec-fce1-4e25-ad9a-5738f1c9f16a None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:35.074307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a33ec-fce1-4e25-ad9a-5738f1c9f16a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:35.075419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a33ec-fce1-4e25-ad9a-5738f1c9f16a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:35.076859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a33ec-fce1-4e25-ad9a-5738f1c9f16a None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:35.078440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 423/842] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:35.084278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa3dbc-60b0-4728-93fa-192bfa13f049 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:35.178972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa3dbc-60b0-4728-93fa-192bfa13f049 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:35.198818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:35.212442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d461eb4-d2ee-45d4-beed-91855ce50dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:35.213070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d461eb4-d2ee-45d4-beed-91855ce50dce None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:35.213347 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d461eb4-d2ee-45d4-beed-91855ce50dce None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:35.213613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d461eb4-d2ee-45d4-beed-91855ce50dce None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:35.214260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d461eb4-d2ee-45d4-beed-91855ce50dce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:35.215715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d461eb4-d2ee-45d4-beed-91855ce50dce None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:35.235943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 420/843] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:28:35 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:37.394809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55b8ce-bc80-4011-ac60-a2dda448edcf None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:37.395352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55b8ce-bc80-4011-ac60-a2dda448edcf None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:37.395352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55b8ce-bc80-4011-ac60-a2dda448edcf None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:37.396675 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 424/844] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:37.401797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59799c9-f421-453d-8046-ac892d0f192d None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:37.402741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59799c9-f421-453d-8046-ac892d0f192d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:37.402920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59799c9-f421-453d-8046-ac892d0f192d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:37.420680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-a59799c9-f421-453d-8046-ac892d0f192d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:37.421896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a59799c9-f421-453d-8046-ac892d0f192d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:37.485816 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 421/845] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:37.752776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55b8ce-bc80-4011-ac60-a2dda448edcf None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:37.783596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:37.854335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55b8ce-bc80-4011-ac60-a2dda448edcf None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:37.888003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521cae-b9eb-4bbf-865a-67f82dfd6907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:37.888622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521cae-b9eb-4bbf-865a-67f82dfd6907 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:37.888923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521cae-b9eb-4bbf-865a-67f82dfd6907 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:37.889202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521cae-b9eb-4bbf-865a-67f82dfd6907 None admin] PATH_INFO: `/v3/roles/04b8e62238cb4198928fd327ef78e6ae` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:37.890572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6521cae-b9eb-4bbf-865a-67f82dfd6907 None admin] RBAC: Authorizing `identity:get_role(role_id=04b8e62238cb4198928fd327ef78e6ae)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:37.891936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6521cae-b9eb-4bbf-865a-67f82dfd6907 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:37.893263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 425/846] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:28:37 2022] GET /identity/v3/roles/04b8e62238cb4198928fd327ef78e6ae => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:37.899107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59799c9-f421-453d-8046-ac892d0f192d None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:37.993762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59799c9-f421-453d-8046-ac892d0f192d None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:38.000714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:38.027766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9e6d0-1f3b-4206-9f84-450c183303db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:38.028317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9e6d0-1f3b-4206-9f84-450c183303db None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:38.028675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9e6d0-1f3b-4206-9f84-450c183303db None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:38.028881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9e6d0-1f3b-4206-9f84-450c183303db None admin] PATH_INFO: `/v3/users/059f2c82c0dc4691acd12f04dd66cfde` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:38.041507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9e6d0-1f3b-4206-9f84-450c183303db None admin] RBAC: Authorizing `identity:get_user(user_id=059f2c82c0dc4691acd12f04dd66cfde)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:38.043534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9e6d0-1f3b-4206-9f84-450c183303db None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:38.045143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 422/847] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:28:37 2022] GET /identity/v3/users/059f2c82c0dc4691acd12f04dd66cfde => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:38.050869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521cae-b9eb-4bbf-865a-67f82dfd6907 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:38.144551 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:38.147450 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521cae-b9eb-4bbf-865a-67f82dfd6907 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:38.181866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d04bd-32f4-4270-a861-2ee7c1c44c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:38.182496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d04bd-32f4-4270-a861-2ee7c1c44c33 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:38.182773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d04bd-32f4-4270-a861-2ee7c1c44c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:38.183040 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d04bd-32f4-4270-a861-2ee7c1c44c33 None admin] PATH_INFO: `/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:38.184186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d04bd-32f4-4270-a861-2ee7c1c44c33 None admin] RBAC: Authorizing `identity:get_project(project_id=cd9ac69fc9d44964b76ec47120c06b9f)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:38.185647 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d04bd-32f4-4270-a861-2ee7c1c44c33 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:38.187263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 426/848] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:28:38 2022] GET /identity/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:38.193171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9e6d0-1f3b-4206-9f84-450c183303db None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:38.295506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9e6d0-1f3b-4206-9f84-450c183303db None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:38.328314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14021476-dde6-4874-96e4-78e9fc1610c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:38.328985 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14021476-dde6-4874-96e4-78e9fc1610c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:38.329267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14021476-dde6-4874-96e4-78e9fc1610c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:38.329540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14021476-dde6-4874-96e4-78e9fc1610c0 None admin] PATH_INFO: `/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f/users/059f2c82c0dc4691acd12f04dd66cfde/roles/04b8e62238cb4198928fd327ef78e6ae` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:38.331646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14021476-dde6-4874-96e4-78e9fc1610c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd9ac69fc9d44964b76ec47120c06b9f, user_id=059f2c82c0dc4691acd12f04dd66cfde, role_id=04b8e62238cb4198928fd327ef78e6ae)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:38.333109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14021476-dde6-4874-96e4-78e9fc1610c0 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:38.346582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 423/849] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:28:38 2022] PUT /identity/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f/users/059f2c82c0dc4691acd12f04dd66cfde/roles/04b8e62238cb4198928fd327ef78e6ae => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:28:40.490479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59874c-4d06-4727-82ac-b6e52d320da8 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:40.490984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59874c-4d06-4727-82ac-b6e52d320da8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:40.490984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59874c-4d06-4727-82ac-b6e52d320da8 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:40.492266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 427/850] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:40.498111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e65116-0068-428b-b7c3-696c85b8143d None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:40.498665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e65116-0068-428b-b7c3-696c85b8143d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:40.498665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e65116-0068-428b-b7c3-696c85b8143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:40.517338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c6e65116-0068-428b-b7c3-696c85b8143d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:40.518554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c6e65116-0068-428b-b7c3-696c85b8143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:40.537260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:40.587063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 424/851] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:40.863446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59874c-4d06-4727-82ac-b6e52d320da8 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:40.965490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59874c-4d06-4727-82ac-b6e52d320da8 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:40.999260 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4860bbc-490c-4f86-bab4-83286c2512b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:40.999894 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4860bbc-490c-4f86-bab4-83286c2512b3 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:41.000173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4860bbc-490c-4f86-bab4-83286c2512b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:41.000443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4860bbc-490c-4f86-bab4-83286c2512b3 None admin] PATH_INFO: `/v3/roles/04b8e62238cb4198928fd327ef78e6ae` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:41.001846 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4860bbc-490c-4f86-bab4-83286c2512b3 None admin] RBAC: Authorizing `identity:get_role(role_id=04b8e62238cb4198928fd327ef78e6ae)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:41.003210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4860bbc-490c-4f86-bab4-83286c2512b3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:41.004541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 428/852] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:28:40 2022] GET /identity/v3/roles/04b8e62238cb4198928fd327ef78e6ae => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:41.010615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e65116-0068-428b-b7c3-696c85b8143d None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:41.106500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e65116-0068-428b-b7c3-696c85b8143d None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:41.140196 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa9647-0594-4fcc-87b7-3f1ec4d85714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:41.140822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa9647-0594-4fcc-87b7-3f1ec4d85714 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:41.141131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa9647-0594-4fcc-87b7-3f1ec4d85714 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:41.141409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa9647-0594-4fcc-87b7-3f1ec4d85714 None admin] PATH_INFO: `/v3/users/059f2c82c0dc4691acd12f04dd66cfde` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:41.142550 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa9647-0594-4fcc-87b7-3f1ec4d85714 None admin] RBAC: Authorizing `identity:get_user(user_id=059f2c82c0dc4691acd12f04dd66cfde)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:41.143983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa9647-0594-4fcc-87b7-3f1ec4d85714 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:41.145581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 425/853] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:28:41 2022] GET /identity/v3/users/059f2c82c0dc4691acd12f04dd66cfde => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:41.151493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4860bbc-490c-4f86-bab4-83286c2512b3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:41.187296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:41.252327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4860bbc-490c-4f86-bab4-83286c2512b3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:41.289339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1809-d467-4ca8-b0ca-8596b4a73998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:41.289951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1809-d467-4ca8-b0ca-8596b4a73998 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:41.290228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1809-d467-4ca8-b0ca-8596b4a73998 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:41.290498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1809-d467-4ca8-b0ca-8596b4a73998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:41.356943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e1809-d467-4ca8-b0ca-8596b4a73998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:41.358386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e1809-d467-4ca8-b0ca-8596b4a73998 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:41.391480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:41.457337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 429/854] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:28:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 18 11:28:41.463441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa9647-0594-4fcc-87b7-3f1ec4d85714 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:41.488023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:41.569336 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa9647-0594-4fcc-87b7-3f1ec4d85714 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:41.603214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edc712-cbe6-4563-a056-1255458c502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:41.603835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edc712-cbe6-4563-a056-1255458c502e None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:41.604108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edc712-cbe6-4563-a056-1255458c502e None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:41.604376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edc712-cbe6-4563-a056-1255458c502e None admin] PATH_INFO: `/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:41.605511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edc712-cbe6-4563-a056-1255458c502e None admin] RBAC: Authorizing `identity:get_project(project_id=cd9ac69fc9d44964b76ec47120c06b9f)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:41.606961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edc712-cbe6-4563-a056-1255458c502e None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:41.608573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 426/855] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:28:41 2022] GET /identity/v3/projects/cd9ac69fc9d44964b76ec47120c06b9f => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:41.615318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1809-d467-4ca8-b0ca-8596b4a73998 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:41.717857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1809-d467-4ca8-b0ca-8596b4a73998 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:41.751074 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c17b4a-d599-4d95-a8d1-033d75782ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:41.751679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c17b4a-d599-4d95-a8d1-033d75782ffb None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:41.751953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c17b4a-d599-4d95-a8d1-033d75782ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:41.752219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c17b4a-d599-4d95-a8d1-033d75782ffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:41.752961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c17b4a-d599-4d95-a8d1-033d75782ffb None admin] RBAC: Adding query filter params (role.id=04b8e62238cb4198928fd327ef78e6ae, scope.project.id=cd9ac69fc9d44964b76ec47120c06b9f, user.id=059f2c82c0dc4691acd12f04dd66cfde) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:41.753265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c17b4a-d599-4d95-a8d1-033d75782ffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:41.754678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c17b4a-d599-4d95-a8d1-033d75782ffb None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:41.757516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:41.761650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 430/856] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:28:41 2022] GET /identity/v3/role_assignments?user.id=059f2c82c0dc4691acd12f04dd66cfde&scope.project.id=cd9ac69fc9d44964b76ec47120c06b9f&role.id=04b8e62238cb4198928fd327ef78e6ae => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:43.918516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2f937-dd3c-4f6c-b6ea-df4954739963 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:43.919081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2f937-dd3c-4f6c-b6ea-df4954739963 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:43.919081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2f937-dd3c-4f6c-b6ea-df4954739963 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:43.920376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 427/857] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:43.925811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46633-a72e-4fe4-9a9f-0934f61146dd None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:43.926214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46633-a72e-4fe4-9a9f-0934f61146dd None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:43.926352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46633-a72e-4fe4-9a9f-0934f61146dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:43.945560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-92f46633-a72e-4fe4-9a9f-0934f61146dd None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:43.946763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-92f46633-a72e-4fe4-9a9f-0934f61146dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:44.014691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 431/858] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:44.284608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2f937-dd3c-4f6c-b6ea-df4954739963 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:44.382353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:44.392948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2f937-dd3c-4f6c-b6ea-df4954739963 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:44.425489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02fc82-b7e8-480d-8fbf-0457831435ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:44.426103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02fc82-b7e8-480d-8fbf-0457831435ce None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:44.426381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02fc82-b7e8-480d-8fbf-0457831435ce None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:44.426648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02fc82-b7e8-480d-8fbf-0457831435ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:44.427747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02fc82-b7e8-480d-8fbf-0457831435ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:44.429232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02fc82-b7e8-480d-8fbf-0457831435ce None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:44.430739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 428/859] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:44.437267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46633-a72e-4fe4-9a9f-0934f61146dd None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:44.533656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46633-a72e-4fe4-9a9f-0934f61146dd None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:44.561792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:44.567879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddc172-f97e-49ed-8858-78a4a3bc70f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:44.568490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddc172-f97e-49ed-8858-78a4a3bc70f5 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:44.568768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddc172-f97e-49ed-8858-78a4a3bc70f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:44.569068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddc172-f97e-49ed-8858-78a4a3bc70f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:44.569728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddc172-f97e-49ed-8858-78a4a3bc70f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:44.571149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddc172-f97e-49ed-8858-78a4a3bc70f5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:44.597731 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 432/860] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:28:44 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:46.775537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef67f9b-b5d9-4f98-9e63-aea5a23e769e None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:46.776208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef67f9b-b5d9-4f98-9e63-aea5a23e769e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:46.776208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef67f9b-b5d9-4f98-9e63-aea5a23e769e None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:46.777960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 429/861] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:46.783331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca965-99cb-482c-923c-085261aa41bb None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:46.783765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca965-99cb-482c-923c-085261aa41bb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:46.783907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca965-99cb-482c-923c-085261aa41bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:46.802483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-087ca965-99cb-482c-923c-085261aa41bb None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:46.803677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-087ca965-99cb-482c-923c-085261aa41bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:46.868468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 433/862] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:47.137455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef67f9b-b5d9-4f98-9e63-aea5a23e769e None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:47.240480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef67f9b-b5d9-4f98-9e63-aea5a23e769e None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:47.273346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f04e72-94ad-45de-881a-71055d94f0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:47.273962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f04e72-94ad-45de-881a-71055d94f0ca None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:47.274236 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f04e72-94ad-45de-881a-71055d94f0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:47.274502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f04e72-94ad-45de-881a-71055d94f0ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:47.275586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f04e72-94ad-45de-881a-71055d94f0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:47.277059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f04e72-94ad-45de-881a-71055d94f0ca None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:47.278583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 430/863] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:28:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:47.284388 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ca965-99cb-482c-923c-085261aa41bb None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:47.385879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ca965-99cb-482c-923c-085261aa41bb None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:47.399596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:47.426170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1937b0-4d22-45af-b26d-21d53ed31f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:47.426785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1937b0-4d22-45af-b26d-21d53ed31f54 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:47.427063 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1937b0-4d22-45af-b26d-21d53ed31f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:47.427337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1937b0-4d22-45af-b26d-21d53ed31f54 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:47.428002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1937b0-4d22-45af-b26d-21d53ed31f54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:47.429470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1937b0-4d22-45af-b26d-21d53ed31f54 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:47.453492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 434/864] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:28:47 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:28:49.608044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091e2b-798c-4a96-aa1c-3f0f9e5c8147 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:49.608554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091e2b-798c-4a96-aa1c-3f0f9e5c8147 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:49.608554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3091e2b-798c-4a96-aa1c-3f0f9e5c8147 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:49.609847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 431/865] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:49.615236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d642b3b-c554-468a-b162-2f4c2f7e6cc1 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:49.615679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d642b3b-c554-468a-b162-2f4c2f7e6cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:49.615783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d642b3b-c554-468a-b162-2f4c2f7e6cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:49.634305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3d642b3b-c554-468a-b162-2f4c2f7e6cc1 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:49.635541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3d642b3b-c554-468a-b162-2f4c2f7e6cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:49.700963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 435/866] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:49.965837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3091e2b-798c-4a96-aa1c-3f0f9e5c8147 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:49.993159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:50.065978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3091e2b-798c-4a96-aa1c-3f0f9e5c8147 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:50.100470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d82dec-b2f7-4ce5-bc0e-5b9b0cf355a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:50.101123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d82dec-b2f7-4ce5-bc0e-5b9b0cf355a3 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:50.101404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d82dec-b2f7-4ce5-bc0e-5b9b0cf355a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:50.101673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d82dec-b2f7-4ce5-bc0e-5b9b0cf355a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:50.111508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d82dec-b2f7-4ce5-bc0e-5b9b0cf355a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-c1d82dec-b2f7-4ce5-bc0e-5b9b0cf355a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:28:50.116538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:50.117782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:28:50.118812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 432/867] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:28:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:50.123793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d642b3b-c554-468a-b162-2f4c2f7e6cc1 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:50.219962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:50.223895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d642b3b-c554-468a-b162-2f4c2f7e6cc1 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:50.259047 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90d0b4-c8b2-4985-a45f-e06c41b9f404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:50.259663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d0b4-c8b2-4985-a45f-e06c41b9f404 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:50.259943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d0b4-c8b2-4985-a45f-e06c41b9f404 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:50.260216 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d0b4-c8b2-4985-a45f-e06c41b9f404 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:50.261419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90d0b4-c8b2-4985-a45f-e06c41b9f404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:50.261724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90d0b4-c8b2-4985-a45f-e06c41b9f404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:50.263104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90d0b4-c8b2-4985-a45f-e06c41b9f404 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:50.271522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 436/868] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:28:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:50.278139 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d82dec-b2f7-4ce5-bc0e-5b9b0cf355a3 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:50.474082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:50.489159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d82dec-b2f7-4ce5-bc0e-5b9b0cf355a3 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:50.521819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529bb3a-7ced-4dca-bb44-81e5d0b895dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:50.522440 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529bb3a-7ced-4dca-bb44-81e5d0b895dc None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:50.522713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529bb3a-7ced-4dca-bb44-81e5d0b895dc None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:50.522979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529bb3a-7ced-4dca-bb44-81e5d0b895dc None admin] PATH_INFO: `/v3/users/873085f9bfdf4e8da7489c6548c0fa03` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:50.536205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529bb3a-7ced-4dca-bb44-81e5d0b895dc None admin] RBAC: Authorizing `identity:get_user(user_id=873085f9bfdf4e8da7489c6548c0fa03)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:50.537661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529bb3a-7ced-4dca-bb44-81e5d0b895dc None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:50.539256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 433/869] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:28:50 2022] GET /identity/v3/users/873085f9bfdf4e8da7489c6548c0fa03 => generated 286 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:50.545719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90d0b4-c8b2-4985-a45f-e06c41b9f404 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:50.644115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90d0b4-c8b2-4985-a45f-e06c41b9f404 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:50.678023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25decb-b76b-4fa0-8ae7-638d35553bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:50.678653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25decb-b76b-4fa0-8ae7-638d35553bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:50.678933 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25decb-b76b-4fa0-8ae7-638d35553bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:50.679203 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25decb-b76b-4fa0-8ae7-638d35553bf1 None admin] PATH_INFO: `/v3/projects/98fc18ca2b3f46e5a26c08008ef8e550` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:50.680314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25decb-b76b-4fa0-8ae7-638d35553bf1 None admin] RBAC: Authorizing `identity:get_project(project_id=98fc18ca2b3f46e5a26c08008ef8e550)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:50.681773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25decb-b76b-4fa0-8ae7-638d35553bf1 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:50.683428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 437/870] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:28:50 2022] GET /identity/v3/projects/98fc18ca2b3f46e5a26c08008ef8e550 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:50.689443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529bb3a-7ced-4dca-bb44-81e5d0b895dc None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:50.787884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529bb3a-7ced-4dca-bb44-81e5d0b895dc None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:50.820683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d65c66-091a-4dad-be67-9af2dcac4f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:50.821358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65c66-091a-4dad-be67-9af2dcac4f9b None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:50.821635 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65c66-091a-4dad-be67-9af2dcac4f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:50.821907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65c66-091a-4dad-be67-9af2dcac4f9b None admin] PATH_INFO: `/v3/projects/98fc18ca2b3f46e5a26c08008ef8e550/users/873085f9bfdf4e8da7489c6548c0fa03/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:50.824278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65c66-091a-4dad-be67-9af2dcac4f9b None admin] RBAC: Authorizing `identity:create_grant(project_id=98fc18ca2b3f46e5a26c08008ef8e550, user_id=873085f9bfdf4e8da7489c6548c0fa03, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:50.825725 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65c66-091a-4dad-be67-9af2dcac4f9b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:50.839628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 434/871] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:28:50 2022] PUT /identity/v3/projects/98fc18ca2b3f46e5a26c08008ef8e550/users/873085f9bfdf4e8da7489c6548c0fa03/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:28:52.993029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1f455-88bf-438c-927b-7e2879b3b9c2 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:52.993885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1f455-88bf-438c-927b-7e2879b3b9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:52.994062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1f455-88bf-438c-927b-7e2879b3b9c2 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:52.995934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 438/872] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:28:53.001903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647487dd-0ac8-484a-a61b-8f698ac0cd1b None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:53.002616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647487dd-0ac8-484a-a61b-8f698ac0cd1b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:53.002616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647487dd-0ac8-484a-a61b-8f698ac0cd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:53.009769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:53.023370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-647487dd-0ac8-484a-a61b-8f698ac0cd1b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:53.024620 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-647487dd-0ac8-484a-a61b-8f698ac0cd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:53.097124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 435/873] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:28:53.368265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1f455-88bf-438c-927b-7e2879b3b9c2 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:53.402660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:53.494489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1f455-88bf-438c-927b-7e2879b3b9c2 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:53.537078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c1495-fcd9-4a14-b749-624aee3d9660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:53.537818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c1495-fcd9-4a14-b749-624aee3d9660 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:53.538097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c1495-fcd9-4a14-b749-624aee3d9660 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:53.538369 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c1495-fcd9-4a14-b749-624aee3d9660 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:53.551535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c1495-fcd9-4a14-b749-624aee3d9660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-234c1495-fcd9-4a14-b749-624aee3d9660 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:28:53.556666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:28:53.559118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:28:53.560690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 439/874] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:28:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:28:53.566542 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647487dd-0ac8-484a-a61b-8f698ac0cd1b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:53.679867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647487dd-0ac8-484a-a61b-8f698ac0cd1b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:53.716765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48b193-eb11-4f71-b04e-b0c506f93387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:53.717453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48b193-eb11-4f71-b04e-b0c506f93387 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:53.717779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48b193-eb11-4f71-b04e-b0c506f93387 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:53.718096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48b193-eb11-4f71-b04e-b0c506f93387 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:53.718816 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48b193-eb11-4f71-b04e-b0c506f93387 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:53.719261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48b193-eb11-4f71-b04e-b0c506f93387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:53.721143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48b193-eb11-4f71-b04e-b0c506f93387 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:53.722900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:53.732217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 436/875] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:28:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:53.738751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c1495-fcd9-4a14-b749-624aee3d9660 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:53.840124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:53.855298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c1495-fcd9-4a14-b749-624aee3d9660 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:53.893786 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a2d95-ba6e-4e2d-bf37-d0ffd63b4709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:53.894501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2d95-ba6e-4e2d-bf37-d0ffd63b4709 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:53.894824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2d95-ba6e-4e2d-bf37-d0ffd63b4709 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:53.895171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2d95-ba6e-4e2d-bf37-d0ffd63b4709 None admin] PATH_INFO: `/v3/users/873085f9bfdf4e8da7489c6548c0fa03` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:53.896643 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a2d95-ba6e-4e2d-bf37-d0ffd63b4709 None admin] RBAC: Authorizing `identity:get_user(user_id=873085f9bfdf4e8da7489c6548c0fa03)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:53.898214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a2d95-ba6e-4e2d-bf37-d0ffd63b4709 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:53.900031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 440/876] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:28:53 2022] GET /identity/v3/users/873085f9bfdf4e8da7489c6548c0fa03 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:53.906495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48b193-eb11-4f71-b04e-b0c506f93387 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:54.022402 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48b193-eb11-4f71-b04e-b0c506f93387 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:54.058818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e512d0-6954-433c-8e74-a1bfa1113e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:54.059433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e512d0-6954-433c-8e74-a1bfa1113e95 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:54.059709 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e512d0-6954-433c-8e74-a1bfa1113e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:54.059976 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e512d0-6954-433c-8e74-a1bfa1113e95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:54.124684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:54.132092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e512d0-6954-433c-8e74-a1bfa1113e95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:54.133580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e512d0-6954-433c-8e74-a1bfa1113e95 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:54.241241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 437/877] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:28:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 18 11:28:54.250574 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a2d95-ba6e-4e2d-bf37-d0ffd63b4709 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:54.367927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a2d95-ba6e-4e2d-bf37-d0ffd63b4709 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:54.404232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fcdcf-b784-45bc-b60a-72f039182026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:54.404892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fcdcf-b784-45bc-b60a-72f039182026 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:54.405173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fcdcf-b784-45bc-b60a-72f039182026 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:54.405444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fcdcf-b784-45bc-b60a-72f039182026 None admin] PATH_INFO: `/v3/projects/98fc18ca2b3f46e5a26c08008ef8e550` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:54.406614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fcdcf-b784-45bc-b60a-72f039182026 None admin] RBAC: Authorizing `identity:get_project(project_id=98fc18ca2b3f46e5a26c08008ef8e550)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:54.408048 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fcdcf-b784-45bc-b60a-72f039182026 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:54.409739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 441/878] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:28:54 2022] GET /identity/v3/projects/98fc18ca2b3f46e5a26c08008ef8e550 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:28:54.416171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e512d0-6954-433c-8e74-a1bfa1113e95 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:54.516349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:54.523889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e512d0-6954-433c-8e74-a1bfa1113e95 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:54.562355 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26162c43-d461-4cf6-ab4b-0fb12d6660dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:54.562973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26162c43-d461-4cf6-ab4b-0fb12d6660dd None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:54.563251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26162c43-d461-4cf6-ab4b-0fb12d6660dd None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:54.563522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26162c43-d461-4cf6-ab4b-0fb12d6660dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:54.564261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26162c43-d461-4cf6-ab4b-0fb12d6660dd None admin] RBAC: Adding query filter params (role.id=8d9ff9da6ad946a6bc49336777182ff9, scope.project.id=98fc18ca2b3f46e5a26c08008ef8e550, user.id=873085f9bfdf4e8da7489c6548c0fa03) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:28:54.564569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26162c43-d461-4cf6-ab4b-0fb12d6660dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:54.566081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26162c43-d461-4cf6-ab4b-0fb12d6660dd None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:54.572727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 438/879] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:28:54 2022] GET /identity/v3/role_assignments?user.id=873085f9bfdf4e8da7489c6548c0fa03&scope.project.id=98fc18ca2b3f46e5a26c08008ef8e550&role.id=8d9ff9da6ad946a6bc49336777182ff9 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:28:56.819116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07b9a5-ae77-41f9-af60-e7d3e1d6933d None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:56.819673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07b9a5-ae77-41f9-af60-e7d3e1d6933d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:56.819673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07b9a5-ae77-41f9-af60-e7d3e1d6933d None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:56.821016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 442/880] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:56.827169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1920b77-0c07-47f0-8448-127db808405c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:56.827714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1920b77-0c07-47f0-8448-127db808405c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:56.827714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1920b77-0c07-47f0-8448-127db808405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:56.846300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-a1920b77-0c07-47f0-8448-127db808405c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:56.847543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a1920b77-0c07-47f0-8448-127db808405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:56.918921 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 439/881] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:57.197814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07b9a5-ae77-41f9-af60-e7d3e1d6933d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:28:57.226203 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:28:57.308110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07b9a5-ae77-41f9-af60-e7d3e1d6933d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:28:57.345438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645704b-0dd0-4427-b726-9393f76d8e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:28:57.346100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645704b-0dd0-4427-b726-9393f76d8e1d None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:57.346416 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645704b-0dd0-4427-b726-9393f76d8e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:57.346726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645704b-0dd0-4427-b726-9393f76d8e1d None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:57.347513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645704b-0dd0-4427-b726-9393f76d8e1d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:28:57.349071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645704b-0dd0-4427-b726-9393f76d8e1d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:28:57.369223 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 443/882] 10.222.0.17 () {60 vars in 1138 bytes} [Tue Oct 18 11:28:57 2022] POST /identity/v3/domains => generated 267 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:28:59.618107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540643-b6b8-4e07-9262-37620874ccb9 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:59.618627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540643-b6b8-4e07-9262-37620874ccb9 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:59.618761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540643-b6b8-4e07-9262-37620874ccb9 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:59.620033 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 440/883] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:28:59.625538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb2ac7-1bdb-4bda-a53d-fdac905f56ad None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:28:59.625969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb2ac7-1bdb-4bda-a53d-fdac905f56ad None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:28:59.626053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb2ac7-1bdb-4bda-a53d-fdac905f56ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:28:59.644142 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e4fb2ac7-1bdb-4bda-a53d-fdac905f56ad None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:28:59.645396 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e4fb2ac7-1bdb-4bda-a53d-fdac905f56ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:28:59.711485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 444/884] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:28:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:28:59.986239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df540643-b6b8-4e07-9262-37620874ccb9 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:00.078371 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:00.089423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df540643-b6b8-4e07-9262-37620874ccb9 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:00.123472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7a110-e0cb-4d15-aca3-774f6d6598b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:00.124099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7a110-e0cb-4d15-aca3-774f6d6598b0 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:00.124375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7a110-e0cb-4d15-aca3-774f6d6598b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:00.124646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7a110-e0cb-4d15-aca3-774f6d6598b0 None admin] PATH_INFO: `/v3/domains/3f18142fb6ac47cf91e70a27c527aed3` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:00.135137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7a110-e0cb-4d15-aca3-774f6d6598b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3f18142fb6ac47cf91e70a27c527aed3)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:00.137149 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7a110-e0cb-4d15-aca3-774f6d6598b0 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:00.138791 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 441/885] 10.222.0.17 () {56 vars in 1183 bytes} [Tue Oct 18 11:28:59 2022] GET /identity/v3/domains/3f18142fb6ac47cf91e70a27c527aed3 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:29:00.144713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb2ac7-1bdb-4bda-a53d-fdac905f56ad None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:00.159860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:00.242985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb2ac7-1bdb-4bda-a53d-fdac905f56ad None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:00.276760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9deb2-cf5f-46d3-98c5-1d2986fe7438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:00.277393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9deb2-cf5f-46d3-98c5-1d2986fe7438 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:00.277670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9deb2-cf5f-46d3-98c5-1d2986fe7438 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:00.277937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9deb2-cf5f-46d3-98c5-1d2986fe7438 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:00.278639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9deb2-cf5f-46d3-98c5-1d2986fe7438 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:00.280105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9deb2-cf5f-46d3-98c5-1d2986fe7438 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:00.308196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 445/886] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:29:00 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:29:02.508459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94664846-2e91-4187-be8a-e9e9283298fb None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:02.509081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94664846-2e91-4187-be8a-e9e9283298fb None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:02.509081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94664846-2e91-4187-be8a-e9e9283298fb None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:02.510831 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 442/887] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:29:02.516175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b37d0-20ca-424b-baa3-1bdb55501da5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:02.516592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b37d0-20ca-424b-baa3-1bdb55501da5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:02.516726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b37d0-20ca-424b-baa3-1bdb55501da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:02.534931 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b89b37d0-20ca-424b-baa3-1bdb55501da5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:29:02.536143 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b89b37d0-20ca-424b-baa3-1bdb55501da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:29:02.596772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:02.605289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 446/888] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:29:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:29:02.880871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94664846-2e91-4187-be8a-e9e9283298fb None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:02.984107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94664846-2e91-4187-be8a-e9e9283298fb None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:03.034466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebd6e1-2773-4c1a-be91-fd5b43f23455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:03.035518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebd6e1-2773-4c1a-be91-fd5b43f23455 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:03.036003 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebd6e1-2773-4c1a-be91-fd5b43f23455 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:03.036452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebd6e1-2773-4c1a-be91-fd5b43f23455 None admin] PATH_INFO: `/v3/domains/3f18142fb6ac47cf91e70a27c527aed3` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:03.038339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebd6e1-2773-4c1a-be91-fd5b43f23455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3f18142fb6ac47cf91e70a27c527aed3)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:03.040973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebd6e1-2773-4c1a-be91-fd5b43f23455 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:03.043762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 443/889] 10.222.0.17 () {56 vars in 1183 bytes} [Tue Oct 18 11:29:02 2022] GET /identity/v3/domains/3f18142fb6ac47cf91e70a27c527aed3 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:29:03.049659 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b37d0-20ca-424b-baa3-1bdb55501da5 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:03.148481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b37d0-20ca-424b-baa3-1bdb55501da5 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:03.186038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8ccb0-7477-402e-bd94-123bcf2fa4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:03.186656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ccb0-7477-402e-bd94-123bcf2fa4af None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:03.186938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ccb0-7477-402e-bd94-123bcf2fa4af None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:03.187214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ccb0-7477-402e-bd94-123bcf2fa4af None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:03.187873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8ccb0-7477-402e-bd94-123bcf2fa4af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:03.189358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8ccb0-7477-402e-bd94-123bcf2fa4af None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:03.211232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 447/890] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:29:03 2022] POST /identity/v3/users => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:29:05.403446 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf93bd4-2554-46f4-8058-107437af732a None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:05.403998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf93bd4-2554-46f4-8058-107437af732a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:05.403998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf93bd4-2554-46f4-8058-107437af732a None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:05.405467 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 444/891] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:29:05.410794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f65e9-c2af-4fbe-a926-3e0752676dfd None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:05.411233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f65e9-c2af-4fbe-a926-3e0752676dfd None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:05.411330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f65e9-c2af-4fbe-a926-3e0752676dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:05.430253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e02f65e9-c2af-4fbe-a926-3e0752676dfd None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:29:05.431375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e02f65e9-c2af-4fbe-a926-3e0752676dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:29:05.478622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:05.501266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 448/892] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:29:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:29:05.770131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf93bd4-2554-46f4-8058-107437af732a None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:05.792388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:05.876153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf93bd4-2554-46f4-8058-107437af732a None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:05.909426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d957dbd-b388-4975-b05e-d8d5719193a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:05.910048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d957dbd-b388-4975-b05e-d8d5719193a0 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:05.910320 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d957dbd-b388-4975-b05e-d8d5719193a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:05.910586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d957dbd-b388-4975-b05e-d8d5719193a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:05.920960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d957dbd-b388-4975-b05e-d8d5719193a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-9d957dbd-b388-4975-b05e-d8d5719193a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:29:05.925713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:29:05.926795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:29:05.927896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 445/893] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:29:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:29:05.933741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f65e9-c2af-4fbe-a926-3e0752676dfd None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:06.036545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f65e9-c2af-4fbe-a926-3e0752676dfd None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:06.075250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58bfae-c096-4327-972a-bedbdd476cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:06.075866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58bfae-c096-4327-972a-bedbdd476cea None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:06.076150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58bfae-c096-4327-972a-bedbdd476cea None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:06.076421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58bfae-c096-4327-972a-bedbdd476cea None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:06.077113 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58bfae-c096-4327-972a-bedbdd476cea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:29:06.077415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58bfae-c096-4327-972a-bedbdd476cea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:06.078789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58bfae-c096-4327-972a-bedbdd476cea None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:06.087072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 449/894] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:29:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:29:06.093806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d957dbd-b388-4975-b05e-d8d5719193a0 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:06.171714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:06.196260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d957dbd-b388-4975-b05e-d8d5719193a0 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:06.228615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85858f9b-9adc-460b-abc0-ed12d545d1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:06.229872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85858f9b-9adc-460b-abc0-ed12d545d1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:06.230149 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85858f9b-9adc-460b-abc0-ed12d545d1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:06.230415 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85858f9b-9adc-460b-abc0-ed12d545d1e2 None admin] PATH_INFO: `/v3/users/ffde259247cd4755897c671341d3dbe7` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:06.243673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85858f9b-9adc-460b-abc0-ed12d545d1e2 None admin] RBAC: Authorizing `identity:get_user(user_id=ffde259247cd4755897c671341d3dbe7)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:06.245215 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85858f9b-9adc-460b-abc0-ed12d545d1e2 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:06.246847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 446/895] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:29:06 2022] GET /identity/v3/users/ffde259247cd4755897c671341d3dbe7 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:29:06.252818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58bfae-c096-4327-972a-bedbdd476cea None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:06.299044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:06.352027 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58bfae-c096-4327-972a-bedbdd476cea None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:06.385972 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a506f-95c2-4a90-b3ab-654e85783bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:06.386605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a506f-95c2-4a90-b3ab-654e85783bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:06.386881 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a506f-95c2-4a90-b3ab-654e85783bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:06.387146 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a506f-95c2-4a90-b3ab-654e85783bf2 None admin] PATH_INFO: `/v3/projects/22cf40ba02d444e687d00317c7119431` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:06.388280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a506f-95c2-4a90-b3ab-654e85783bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=22cf40ba02d444e687d00317c7119431)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:06.389731 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a506f-95c2-4a90-b3ab-654e85783bf2 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:06.391377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 450/896] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:29:06 2022] GET /identity/v3/projects/22cf40ba02d444e687d00317c7119431 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:29:06.397574 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85858f9b-9adc-460b-abc0-ed12d545d1e2 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:06.499112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85858f9b-9adc-460b-abc0-ed12d545d1e2 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:06.533115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7216bb6-f19f-4510-961e-043319af66e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:06.533769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7216bb6-f19f-4510-961e-043319af66e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:06.534042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7216bb6-f19f-4510-961e-043319af66e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:06.534307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7216bb6-f19f-4510-961e-043319af66e9 None admin] PATH_INFO: `/v3/projects/22cf40ba02d444e687d00317c7119431/users/ffde259247cd4755897c671341d3dbe7/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:06.536185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7216bb6-f19f-4510-961e-043319af66e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=22cf40ba02d444e687d00317c7119431, user_id=ffde259247cd4755897c671341d3dbe7, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:06.537631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7216bb6-f19f-4510-961e-043319af66e9 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:06.540816 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:06.553672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 447/897] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:29:06 2022] PUT /identity/v3/projects/22cf40ba02d444e687d00317c7119431/users/ffde259247cd4755897c671341d3dbe7/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:29:08.705326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64e01c-6229-471b-aa20-fe83b3a9e516 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:08.705852 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64e01c-6229-471b-aa20-fe83b3a9e516 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:08.705852 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64e01c-6229-471b-aa20-fe83b3a9e516 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:08.707185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 451/898] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:29:08.712322 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ad498-6c39-4451-aaec-5d9bf9c34984 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:08.712715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ad498-6c39-4451-aaec-5d9bf9c34984 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:08.712884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ad498-6c39-4451-aaec-5d9bf9c34984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:08.731873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-f84ad498-6c39-4451-aaec-5d9bf9c34984 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:29:08.733088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f84ad498-6c39-4451-aaec-5d9bf9c34984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:29:08.802101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 448/899] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:29:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:29:09.069946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64e01c-6229-471b-aa20-fe83b3a9e516 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:09.174986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64e01c-6229-471b-aa20-fe83b3a9e516 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:09.188606 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:09.209793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e5215-de70-47d7-a8ae-ee6ac6a2226f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:09.210418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e5215-de70-47d7-a8ae-ee6ac6a2226f None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:09.210695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e5215-de70-47d7-a8ae-ee6ac6a2226f None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:09.210967 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e5215-de70-47d7-a8ae-ee6ac6a2226f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:09.222951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e5215-de70-47d7-a8ae-ee6ac6a2226f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-0d4e5215-de70-47d7-a8ae-ee6ac6a2226f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:29:09.227778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:29:09.229448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:29:09.230376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 452/900] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:29:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:29:09.235526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ad498-6c39-4451-aaec-5d9bf9c34984 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:09.330788 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ad498-6c39-4451-aaec-5d9bf9c34984 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:09.356273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:09.365557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375e422-a09a-41fc-b8e0-c617c086bf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:09.366163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375e422-a09a-41fc-b8e0-c617c086bf46 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:09.366480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375e422-a09a-41fc-b8e0-c617c086bf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:09.366748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375e422-a09a-41fc-b8e0-c617c086bf46 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:09.367406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375e422-a09a-41fc-b8e0-c617c086bf46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:29:09.367702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375e422-a09a-41fc-b8e0-c617c086bf46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:09.369084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375e422-a09a-41fc-b8e0-c617c086bf46 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:09.377237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 449/901] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:29:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:29:09.383934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e5215-de70-47d7-a8ae-ee6ac6a2226f None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:09.481945 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e5215-de70-47d7-a8ae-ee6ac6a2226f None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:09.515007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599617aa-55fc-4ff3-a117-f4f38bb8cb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:09.515632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599617aa-55fc-4ff3-a117-f4f38bb8cb34 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:09.515911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599617aa-55fc-4ff3-a117-f4f38bb8cb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:09.516188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599617aa-55fc-4ff3-a117-f4f38bb8cb34 None admin] PATH_INFO: `/v3/users/ffde259247cd4755897c671341d3dbe7` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:09.517330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599617aa-55fc-4ff3-a117-f4f38bb8cb34 None admin] RBAC: Authorizing `identity:get_user(user_id=ffde259247cd4755897c671341d3dbe7)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:09.518775 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599617aa-55fc-4ff3-a117-f4f38bb8cb34 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:09.520354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 453/902] 10.222.0.17 () {56 vars in 1177 bytes} [Tue Oct 18 11:29:09 2022] GET /identity/v3/users/ffde259247cd4755897c671341d3dbe7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:29:09.526670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375e422-a09a-41fc-b8e0-c617c086bf46 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:09.623969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375e422-a09a-41fc-b8e0-c617c086bf46 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:09.656909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533c4ab-998f-43b2-bb35-dc24ebebfec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:09.657513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533c4ab-998f-43b2-bb35-dc24ebebfec4 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:09.657790 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533c4ab-998f-43b2-bb35-dc24ebebfec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:09.658058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533c4ab-998f-43b2-bb35-dc24ebebfec4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:09.712672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:09.725433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533c4ab-998f-43b2-bb35-dc24ebebfec4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:09.726875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533c4ab-998f-43b2-bb35-dc24ebebfec4 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:09.820759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 450/903] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:29:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 18 11:29:09.827247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599617aa-55fc-4ff3-a117-f4f38bb8cb34 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:09.868041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:09.931172 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599617aa-55fc-4ff3-a117-f4f38bb8cb34 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:09.966876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e088a-4c98-479d-93a2-417db0c89537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:09.967510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e088a-4c98-479d-93a2-417db0c89537 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:09.967782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e088a-4c98-479d-93a2-417db0c89537 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:09.968048 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e088a-4c98-479d-93a2-417db0c89537 None admin] PATH_INFO: `/v3/projects/22cf40ba02d444e687d00317c7119431` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:09.969179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e088a-4c98-479d-93a2-417db0c89537 None admin] RBAC: Authorizing `identity:get_project(project_id=22cf40ba02d444e687d00317c7119431)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:09.970623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e088a-4c98-479d-93a2-417db0c89537 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:09.972236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 454/904] 10.222.0.17 () {56 vars in 1186 bytes} [Tue Oct 18 11:29:09 2022] GET /identity/v3/projects/22cf40ba02d444e687d00317c7119431 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:29:09.978943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533c4ab-998f-43b2-bb35-dc24ebebfec4 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:29:10.065543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:10.078864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533c4ab-998f-43b2-bb35-dc24ebebfec4 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:29:10.112964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a585afd-9fc2-4aad-a699-aab968e8eb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:29:10.113567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a585afd-9fc2-4aad-a699-aab968e8eb9f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:29:10.113847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a585afd-9fc2-4aad-a699-aab968e8eb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:29:10.114115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a585afd-9fc2-4aad-a699-aab968e8eb9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:29:10.114836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a585afd-9fc2-4aad-a699-aab968e8eb9f None admin] RBAC: Adding query filter params (role.id=8d9ff9da6ad946a6bc49336777182ff9, scope.project.id=22cf40ba02d444e687d00317c7119431, user.id=ffde259247cd4755897c671341d3dbe7) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:29:10.115143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a585afd-9fc2-4aad-a699-aab968e8eb9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:29:10.116568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a585afd-9fc2-4aad-a699-aab968e8eb9f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:29:10.122993 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 451/905] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:29:09 2022] GET /identity/v3/role_assignments?user.id=ffde259247cd4755897c671341d3dbe7&scope.project.id=22cf40ba02d444e687d00317c7119431&role.id=8d9ff9da6ad946a6bc49336777182ff9 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:29:19.963498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=46 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:29:20.120336 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=24 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:07.070752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864851c-3265-4e1d-869a-09ecf35e6567 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:07.071575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864851c-3265-4e1d-869a-09ecf35e6567 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:07.071575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864851c-3265-4e1d-869a-09ecf35e6567 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:07.072885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 455/906] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:30:07.080158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a773797-aad7-4453-97b6-ab4280f646e1 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:07.080719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a773797-aad7-4453-97b6-ab4280f646e1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:07.080719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a773797-aad7-4453-97b6-ab4280f646e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:07.111687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1a773797-aad7-4453-97b6-ab4280f646e1 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:30:07.112902 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1a773797-aad7-4453-97b6-ab4280f646e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:30:07.224159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 452/907] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:30:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:30:07.526935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864851c-3265-4e1d-869a-09ecf35e6567 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:07.750943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864851c-3265-4e1d-869a-09ecf35e6567 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:07.815045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064308b-f45a-4f09-ac2c-e8149fd7cdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:07.816141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064308b-f45a-4f09-ac2c-e8149fd7cdda None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:07.816723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064308b-f45a-4f09-ac2c-e8149fd7cdda None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:07.817337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064308b-f45a-4f09-ac2c-e8149fd7cdda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:07.838386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064308b-f45a-4f09-ac2c-e8149fd7cdda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:07.841337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064308b-f45a-4f09-ac2c-e8149fd7cdda None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-a064308b-f45a-4f09-ac2c-e8149fd7cdda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:30:07.865170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:07.867719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:30:07.869701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:30:07.869701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:07.869701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:07.871818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 456/908] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:30:07.874469 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a773797-aad7-4453-97b6-ab4280f646e1 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:08.083113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a773797-aad7-4453-97b6-ab4280f646e1 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:08.160288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:08.177213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34496372-b606-492f-bbf9-1fa79e0b11a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:08.177213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34496372-b606-492f-bbf9-1fa79e0b11a6 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:08.177213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34496372-b606-492f-bbf9-1fa79e0b11a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:08.177213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34496372-b606-492f-bbf9-1fa79e0b11a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:08.177859 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34496372-b606-492f-bbf9-1fa79e0b11a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:08.177859 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34496372-b606-492f-bbf9-1fa79e0b11a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:08.178381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34496372-b606-492f-bbf9-1fa79e0b11a6 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:08.198776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 453/909] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:30:08.202881 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064308b-f45a-4f09-ac2c-e8149fd7cdda None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:08.298488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:08.414645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064308b-f45a-4f09-ac2c-e8149fd7cdda None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:08.499043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6c6ed-3ead-4a6a-b3db-ea6421b8623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:08.501289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6c6ed-3ead-4a6a-b3db-ea6421b8623a None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:08.501717 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6c6ed-3ead-4a6a-b3db-ea6421b8623a None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:08.502208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6c6ed-3ead-4a6a-b3db-ea6421b8623a None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:08.502818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6c6ed-3ead-4a6a-b3db-ea6421b8623a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:08.504685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6c6ed-3ead-4a6a-b3db-ea6421b8623a None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:08.542960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 457/910] 10.222.0.17 () {60 vars in 1132 bytes} [Tue Oct 18 11:30:08 2022] POST /identity/v3/users => generated 251 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:30:11.056382 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c82366-346f-4f90-a044-feae3abeb554 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:11.056997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c82366-346f-4f90-a044-feae3abeb554 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:11.056997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c82366-346f-4f90-a044-feae3abeb554 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:11.059032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 454/911] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:30:11.063918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b74c5-cbf6-4ed0-b88a-b1589d127292 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:11.064463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b74c5-cbf6-4ed0-b88a-b1589d127292 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:11.064463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b74c5-cbf6-4ed0-b88a-b1589d127292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:11.083386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ba5b74c5-cbf6-4ed0-b88a-b1589d127292 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:30:11.084592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ba5b74c5-cbf6-4ed0-b88a-b1589d127292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:30:11.159916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 458/912] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:30:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:30:11.435092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c82366-346f-4f90-a044-feae3abeb554 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:11.566212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c82366-346f-4f90-a044-feae3abeb554 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:11.626365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743ecb1-78db-4c1f-b973-0f6c344f779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:11.627538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743ecb1-78db-4c1f-b973-0f6c344f779d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:11.627538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743ecb1-78db-4c1f-b973-0f6c344f779d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:11.627538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743ecb1-78db-4c1f-b973-0f6c344f779d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:11.639835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743ecb1-78db-4c1f-b973-0f6c344f779d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-c743ecb1-78db-4c1f-b973-0f6c344f779d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:30:11.644811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:11.645875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:11.646707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 455/913] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:30:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:30:11.654689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b74c5-cbf6-4ed0-b88a-b1589d127292 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:11.705833 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:11.793182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b74c5-cbf6-4ed0-b88a-b1589d127292 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:11.852446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5392c5e-6eae-47d6-8efe-61dba8c501f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:11.853427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5392c5e-6eae-47d6-8efe-61dba8c501f4 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:11.854255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5392c5e-6eae-47d6-8efe-61dba8c501f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:11.854255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5392c5e-6eae-47d6-8efe-61dba8c501f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:11.854928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5392c5e-6eae-47d6-8efe-61dba8c501f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:11.855371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5392c5e-6eae-47d6-8efe-61dba8c501f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:11.856880 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5392c5e-6eae-47d6-8efe-61dba8c501f4 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:11.866595 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 459/914] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:30:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:11.873422 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743ecb1-78db-4c1f-b973-0f6c344f779d None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:11.948759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:12.022015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743ecb1-78db-4c1f-b973-0f6c344f779d None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:12.069087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e41bd-36ba-4edd-9c4b-63cc142aad0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:12.070742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e41bd-36ba-4edd-9c4b-63cc142aad0e None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:12.071170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e41bd-36ba-4edd-9c4b-63cc142aad0e None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:12.071634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e41bd-36ba-4edd-9c4b-63cc142aad0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:12.087293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e41bd-36ba-4edd-9c4b-63cc142aad0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:12.089027 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e41bd-36ba-4edd-9c4b-63cc142aad0e None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-fc7e41bd-36ba-4edd-9c4b-63cc142aad0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:30:12.106741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:12.108225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:30:12.109184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:30:12.109184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:12.109184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:12.109184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 456/915] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:30:12.115333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5392c5e-6eae-47d6-8efe-61dba8c501f4 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:12.255393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:12.260426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5392c5e-6eae-47d6-8efe-61dba8c501f4 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:12.304091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b5360-95d3-46b4-9694-62e5b9b67c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:12.304778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b5360-95d3-46b4-9694-62e5b9b67c41 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:12.305191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b5360-95d3-46b4-9694-62e5b9b67c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:12.305375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b5360-95d3-46b4-9694-62e5b9b67c41 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:12.306030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b5360-95d3-46b4-9694-62e5b9b67c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:12.306333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b5360-95d3-46b4-9694-62e5b9b67c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:12.307699 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b5360-95d3-46b4-9694-62e5b9b67c41 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:12.325152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 460/916] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:12.331482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e41bd-36ba-4edd-9c4b-63cc142aad0e None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:12.462127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e41bd-36ba-4edd-9c4b-63cc142aad0e None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:12.463167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:12.509864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae7219-fc16-41df-adbe-f2d93aa8b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:12.510546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae7219-fc16-41df-adbe-f2d93aa8b626 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:12.510830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae7219-fc16-41df-adbe-f2d93aa8b626 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:12.511103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae7219-fc16-41df-adbe-f2d93aa8b626 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:12.520541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae7219-fc16-41df-adbe-f2d93aa8b626 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:12.522177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae7219-fc16-41df-adbe-f2d93aa8b626 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-52ae7219-fc16-41df-adbe-f2d93aa8b626 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:30:12.535933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:12.537224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:12.539300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:12.539300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:12.539300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:30:12.539300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:30:12.539300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:30:12.539300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:30:12.539300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:30:12.539300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:30:12.539300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 18 11:30:12.539300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:12.541861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 457/917] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:30:12.548522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b5360-95d3-46b4-9694-62e5b9b67c41 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:12.687676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b5360-95d3-46b4-9694-62e5b9b67c41 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:12.731162 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a0c53-5778-4348-a5ee-37ac62cdbdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:12.732770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0c53-5778-4348-a5ee-37ac62cdbdd4 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:12.732770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0c53-5778-4348-a5ee-37ac62cdbdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:12.732770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a0c53-5778-4348-a5ee-37ac62cdbdd4 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:12.733525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a0c53-5778-4348-a5ee-37ac62cdbdd4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:12.733921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a0c53-5778-4348-a5ee-37ac62cdbdd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:12.735693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a0c53-5778-4348-a5ee-37ac62cdbdd4 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:12.757169 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 461/918] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Oct 18 11:30:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:30:12.763572 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae7219-fc16-41df-adbe-f2d93aa8b626 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:12.891589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae7219-fc16-41df-adbe-f2d93aa8b626 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:12.930934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be53ae-5a5b-44b1-ad26-4c5c771b92a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:12.931893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be53ae-5a5b-44b1-ad26-4c5c771b92a2 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:12.932172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be53ae-5a5b-44b1-ad26-4c5c771b92a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:12.932493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be53ae-5a5b-44b1-ad26-4c5c771b92a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:12.944133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be53ae-5a5b-44b1-ad26-4c5c771b92a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:12.945597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be53ae-5a5b-44b1-ad26-4c5c771b92a2 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-b0be53ae-5a5b-44b1-ad26-4c5c771b92a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:30:12.963820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:12.964918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:30:12.972029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:30:12.972029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:12.972029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:12.972029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 458/919] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:12.972320 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a0c53-5778-4348-a5ee-37ac62cdbdd4 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:12.996424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:13.111408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a0c53-5778-4348-a5ee-37ac62cdbdd4 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:13.164027 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a8f38-42a2-4f87-b8a7-55e28bf40686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:13.168224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a8f38-42a2-4f87-b8a7-55e28bf40686 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:13.168766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a8f38-42a2-4f87-b8a7-55e28bf40686 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:13.169208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a8f38-42a2-4f87-b8a7-55e28bf40686 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:13.170104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a8f38-42a2-4f87-b8a7-55e28bf40686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:13.170916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a8f38-42a2-4f87-b8a7-55e28bf40686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:13.172609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a8f38-42a2-4f87-b8a7-55e28bf40686 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:13.196974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 462/920] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:13.201969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be53ae-5a5b-44b1-ad26-4c5c771b92a2 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:13.228072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:13.337510 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be53ae-5a5b-44b1-ad26-4c5c771b92a2 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:13.372611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d4d0e-7dee-49c5-8000-d48ac8f79770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:13.373370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d4d0e-7dee-49c5-8000-d48ac8f79770 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:13.373706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d4d0e-7dee-49c5-8000-d48ac8f79770 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:13.374064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d4d0e-7dee-49c5-8000-d48ac8f79770 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:13.383476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d4d0e-7dee-49c5-8000-d48ac8f79770 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:13.383476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d4d0e-7dee-49c5-8000-d48ac8f79770 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-5a5d4d0e-7dee-49c5-8000-d48ac8f79770 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:30:13.394840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:13.396169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:13.397910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:30:13.397910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:30:13.397910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:30:13.397910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:30:13.397910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:30:13.397910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:30:13.397910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:30:13.397910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:13.397910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 459/921] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:30:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:13.403163 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a8f38-42a2-4f87-b8a7-55e28bf40686 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:13.508200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:13.538022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a8f38-42a2-4f87-b8a7-55e28bf40686 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:13.579132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2c607-54a8-44f0-820e-2363e0fad02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:13.580083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c607-54a8-44f0-820e-2363e0fad02d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:13.580373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c607-54a8-44f0-820e-2363e0fad02d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:13.580659 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c607-54a8-44f0-820e-2363e0fad02d None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:13.581801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2c607-54a8-44f0-820e-2363e0fad02d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:13.582412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2c607-54a8-44f0-820e-2363e0fad02d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:13.584580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2c607-54a8-44f0-820e-2363e0fad02d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:13.605187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 463/922] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:13.606553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d4d0e-7dee-49c5-8000-d48ac8f79770 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:13.691361 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:13.722263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d4d0e-7dee-49c5-8000-d48ac8f79770 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:13.771365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad705066-72f1-4bf1-b010-2dd2772ec3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:13.772028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad705066-72f1-4bf1-b010-2dd2772ec3a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:13.772331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad705066-72f1-4bf1-b010-2dd2772ec3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:13.772605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad705066-72f1-4bf1-b010-2dd2772ec3a1 None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/99ec4a4276f345c2998723e554ab9608/roles/f94c9d39cd6244b4b0a727e200ff2e94` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:13.788359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad705066-72f1-4bf1-b010-2dd2772ec3a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=99ec4a4276f345c2998723e554ab9608, role_id=f94c9d39cd6244b4b0a727e200ff2e94)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:13.789917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad705066-72f1-4bf1-b010-2dd2772ec3a1 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:13.807887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 460/923] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:30:13 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/99ec4a4276f345c2998723e554ab9608/roles/f94c9d39cd6244b4b0a727e200ff2e94 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:30:16.182686 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470c011-66b4-463c-9720-28c809265472 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:16.182686 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470c011-66b4-463c-9720-28c809265472 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:16.182686 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470c011-66b4-463c-9720-28c809265472 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:16.182686 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 464/924] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:30:16.190244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d535050-4fd5-45d9-bfd1-ee41f4681d29 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:16.190729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d535050-4fd5-45d9-bfd1-ee41f4681d29 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:16.190973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d535050-4fd5-45d9-bfd1-ee41f4681d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:16.213852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-8d535050-4fd5-45d9-bfd1-ee41f4681d29 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:30:16.215215 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8d535050-4fd5-45d9-bfd1-ee41f4681d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:30:16.298253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 461/925] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:30:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:30:16.586996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470c011-66b4-463c-9720-28c809265472 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:16.721056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470c011-66b4-463c-9720-28c809265472 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:16.787307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bea31-e7d3-4de9-ac7d-a16f9ed6f747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:16.788921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bea31-e7d3-4de9-ac7d-a16f9ed6f747 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:16.789946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bea31-e7d3-4de9-ac7d-a16f9ed6f747 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:16.789946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bea31-e7d3-4de9-ac7d-a16f9ed6f747 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:16.801318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:16.807284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bea31-e7d3-4de9-ac7d-a16f9ed6f747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-b76bea31-e7d3-4de9-ac7d-a16f9ed6f747 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:30:16.814423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:16.815797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:16.816812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 465/926] 10.222.0.17 () {56 vars in 1102 bytes} [Tue Oct 18 11:30:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:16.825370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d535050-4fd5-45d9-bfd1-ee41f4681d29 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:16.905476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:16.961889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d535050-4fd5-45d9-bfd1-ee41f4681d29 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:17.004624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8f9d0-aa38-4688-bb3e-9249740617e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:17.005287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8f9d0-aa38-4688-bb3e-9249740617e1 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:17.005565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8f9d0-aa38-4688-bb3e-9249740617e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:17.005820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8f9d0-aa38-4688-bb3e-9249740617e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:17.006530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8f9d0-aa38-4688-bb3e-9249740617e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:17.006882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8f9d0-aa38-4688-bb3e-9249740617e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:17.008301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8f9d0-aa38-4688-bb3e-9249740617e1 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:17.018118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 462/927] 10.222.0.17 () {56 vars in 1103 bytes} [Tue Oct 18 11:30:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:17.025756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bea31-e7d3-4de9-ac7d-a16f9ed6f747 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:17.162759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bea31-e7d3-4de9-ac7d-a16f9ed6f747 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:17.221940 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d49cb-b219-4b8b-b82c-b8983070912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:17.222954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d49cb-b219-4b8b-b82c-b8983070912b None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:17.223329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d49cb-b219-4b8b-b82c-b8983070912b None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:17.223701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d49cb-b219-4b8b-b82c-b8983070912b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:17.236058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d49cb-b219-4b8b-b82c-b8983070912b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:17.237872 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d49cb-b219-4b8b-b82c-b8983070912b None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-fc9d49cb-b219-4b8b-b82c-b8983070912b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:30:17.265227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:17.267535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:30:17.269262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:30:17.269262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:17.269262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:17.269262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 466/928] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:17.274012 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8f9d0-aa38-4688-bb3e-9249740617e1 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:17.421981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8f9d0-aa38-4688-bb3e-9249740617e1 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:17.443590 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:17.467241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3d0fc-b632-4303-bed1-fdeba20bf592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:17.468087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3d0fc-b632-4303-bed1-fdeba20bf592 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:17.468539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3d0fc-b632-4303-bed1-fdeba20bf592 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:17.468939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3d0fc-b632-4303-bed1-fdeba20bf592 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:17.469961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3d0fc-b632-4303-bed1-fdeba20bf592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:17.470240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3d0fc-b632-4303-bed1-fdeba20bf592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:17.472077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3d0fc-b632-4303-bed1-fdeba20bf592 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:17.487238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 463/929] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:17.494632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d49cb-b219-4b8b-b82c-b8983070912b None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:17.581804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:17.667282 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d49cb-b219-4b8b-b82c-b8983070912b None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:17.715212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c3bbc-fea9-4e5b-bcc0-9418ee3f3404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:17.715950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3bbc-fea9-4e5b-bcc0-9418ee3f3404 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:17.716322 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3bbc-fea9-4e5b-bcc0-9418ee3f3404 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:17.716667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3bbc-fea9-4e5b-bcc0-9418ee3f3404 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:17.732400 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c3bbc-fea9-4e5b-bcc0-9418ee3f3404 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:17.734643 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c3bbc-fea9-4e5b-bcc0-9418ee3f3404 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-f00c3bbc-fea9-4e5b-bcc0-9418ee3f3404 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:30:17.749383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:17.750835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:17.752118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 467/930] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:17.759271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3d0fc-b632-4303-bed1-fdeba20bf592 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:17.927594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3d0fc-b632-4303-bed1-fdeba20bf592 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:17.971855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc06157-aa6c-4377-9958-6ff15cd3fdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:17.972573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06157-aa6c-4377-9958-6ff15cd3fdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:17.973028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06157-aa6c-4377-9958-6ff15cd3fdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:17.973417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06157-aa6c-4377-9958-6ff15cd3fdf2 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:17.974488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc06157-aa6c-4377-9958-6ff15cd3fdf2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:17.974884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc06157-aa6c-4377-9958-6ff15cd3fdf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:17.976643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc06157-aa6c-4377-9958-6ff15cd3fdf2 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:18.004144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 464/931] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Oct 18 11:30:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:30:18.006168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c3bbc-fea9-4e5b-bcc0-9418ee3f3404 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:18.165514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c3bbc-fea9-4e5b-bcc0-9418ee3f3404 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:18.184575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:18.230123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be10dcf-88d2-4800-9eca-4f2bb090b3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:18.231235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be10dcf-88d2-4800-9eca-4f2bb090b3fa None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:18.231462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be10dcf-88d2-4800-9eca-4f2bb090b3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:18.231834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be10dcf-88d2-4800-9eca-4f2bb090b3fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:18.332455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be10dcf-88d2-4800-9eca-4f2bb090b3fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:18.334271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be10dcf-88d2-4800-9eca-4f2bb090b3fa None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:18.521175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 468/932] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:30:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 18 11:30:18.523131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc06157-aa6c-4377-9958-6ff15cd3fdf2 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:18.567390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:18.660162 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc06157-aa6c-4377-9958-6ff15cd3fdf2 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:18.714501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22a3fb-8397-4d3e-b201-b7ceb6a66191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:18.715299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22a3fb-8397-4d3e-b201-b7ceb6a66191 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:18.715684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22a3fb-8397-4d3e-b201-b7ceb6a66191 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:18.716082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22a3fb-8397-4d3e-b201-b7ceb6a66191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:18.730270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22a3fb-8397-4d3e-b201-b7ceb6a66191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:18.732472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22a3fb-8397-4d3e-b201-b7ceb6a66191 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-6e22a3fb-8397-4d3e-b201-b7ceb6a66191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:30:18.750624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:18.752893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:30:18.754856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:30:18.754856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:18.754856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:18.754856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 465/933] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:30:18.760825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be10dcf-88d2-4800-9eca-4f2bb090b3fa None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:18.777108 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:18.908617 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be10dcf-88d2-4800-9eca-4f2bb090b3fa None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:18.962568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b85c7f-371d-4344-b116-214b59ac7478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:18.963428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b85c7f-371d-4344-b116-214b59ac7478 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:18.963852 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b85c7f-371d-4344-b116-214b59ac7478 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:18.964285 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b85c7f-371d-4344-b116-214b59ac7478 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:18.965330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b85c7f-371d-4344-b116-214b59ac7478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:18.965756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b85c7f-371d-4344-b116-214b59ac7478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:18.967851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b85c7f-371d-4344-b116-214b59ac7478 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:18.985019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 469/934] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:18.993155 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22a3fb-8397-4d3e-b201-b7ceb6a66191 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:19.125355 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:19.159971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22a3fb-8397-4d3e-b201-b7ceb6a66191 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:19.210639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74e666-a355-4538-b041-7237dadad32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:19.212059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74e666-a355-4538-b041-7237dadad32f None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:19.212691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74e666-a355-4538-b041-7237dadad32f None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:19.213222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74e666-a355-4538-b041-7237dadad32f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:19.228399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74e666-a355-4538-b041-7237dadad32f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:19.232066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74e666-a355-4538-b041-7237dadad32f None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-fe74e666-a355-4538-b041-7237dadad32f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:30:19.259453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:19.260622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:19.262555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:30:19.262555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:30:19.262555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:30:19.262555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:30:19.262555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:30:19.262555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:30:19.262555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:30:19.262555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:19.262555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 466/935] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:19.269350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b85c7f-371d-4344-b116-214b59ac7478 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:19.395125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:19.456871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b85c7f-371d-4344-b116-214b59ac7478 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:19.512102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fe734-43ef-4a9f-9961-57d5d14a52a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:19.512824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fe734-43ef-4a9f-9961-57d5d14a52a2 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:19.513216 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fe734-43ef-4a9f-9961-57d5d14a52a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:19.513576 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fe734-43ef-4a9f-9961-57d5d14a52a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:19.514378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fe734-43ef-4a9f-9961-57d5d14a52a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:19.514737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fe734-43ef-4a9f-9961-57d5d14a52a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:19.516330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fe734-43ef-4a9f-9961-57d5d14a52a2 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:19.533062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 470/936] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:30:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:19.541198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74e666-a355-4538-b041-7237dadad32f None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:19.703167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74e666-a355-4538-b041-7237dadad32f None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:19.757515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b9391-06d5-44d4-9b81-383ceaf94285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:19.758501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b9391-06d5-44d4-9b81-383ceaf94285 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:19.758986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b9391-06d5-44d4-9b81-383ceaf94285 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:19.759420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b9391-06d5-44d4-9b81-383ceaf94285 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:19.760521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b9391-06d5-44d4-9b81-383ceaf94285 None admin] RBAC: Adding query filter params (role.id=f94c9d39cd6244b4b0a727e200ff2e94, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=99ec4a4276f345c2998723e554ab9608) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:19.761063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b9391-06d5-44d4-9b81-383ceaf94285 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:19.763071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b9391-06d5-44d4-9b81-383ceaf94285 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:19.770232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 467/937] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:30:19 2022] GET /identity/v3/role_assignments?user.id=99ec4a4276f345c2998723e554ab9608&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=f94c9d39cd6244b4b0a727e200ff2e94 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:22.099530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6b32e-5e19-4b60-9ac1-02c7d481b479 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:22.100182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6b32e-5e19-4b60-9ac1-02c7d481b479 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:22.100182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6b32e-5e19-4b60-9ac1-02c7d481b479 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:22.101911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 471/938] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:30:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:30:22.108709 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a3c43-57cd-4da0-870c-564e2b15f56a None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:22.109599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a3c43-57cd-4da0-870c-564e2b15f56a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:22.109698 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a3c43-57cd-4da0-870c-564e2b15f56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:22.141037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-464a3c43-57cd-4da0-870c-564e2b15f56a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:30:22.143235 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-464a3c43-57cd-4da0-870c-564e2b15f56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:30:22.148409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:22.253339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 468/939] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:30:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:30:22.530407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6b32e-5e19-4b60-9ac1-02c7d481b479 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:22.697359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6b32e-5e19-4b60-9ac1-02c7d481b479 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:22.742489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:22.751306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec6212-bbdf-4166-843b-f7ff8e372be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:22.751986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec6212-bbdf-4166-843b-f7ff8e372be1 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:22.752279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec6212-bbdf-4166-843b-f7ff8e372be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:22.752540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec6212-bbdf-4166-843b-f7ff8e372be1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:22.764990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec6212-bbdf-4166-843b-f7ff8e372be1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-a8ec6212-bbdf-4166-843b-f7ff8e372be1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:30:22.769864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:22.770874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:22.771760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 472/940] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:30:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:22.778676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a3c43-57cd-4da0-870c-564e2b15f56a None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:22.931392 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a3c43-57cd-4da0-870c-564e2b15f56a None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:22.970783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c17bd-b44f-494b-80d9-944d013fe1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:22.971404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c17bd-b44f-494b-80d9-944d013fe1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:22.971719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c17bd-b44f-494b-80d9-944d013fe1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:22.972015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c17bd-b44f-494b-80d9-944d013fe1a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:22.972703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c17bd-b44f-494b-80d9-944d013fe1a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:22.973033 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c17bd-b44f-494b-80d9-944d013fe1a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:22.974491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c17bd-b44f-494b-80d9-944d013fe1a3 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:22.990176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec6212-bbdf-4166-843b-f7ff8e372be1 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:22.991159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 469/941] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:30:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:23.120939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec6212-bbdf-4166-843b-f7ff8e372be1 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:23.159662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631ffde-5538-485a-b677-6e20367ae76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:23.160311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ffde-5538-485a-b677-6e20367ae76c None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:23.160596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ffde-5538-485a-b677-6e20367ae76c None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:23.160892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ffde-5538-485a-b677-6e20367ae76c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:23.174131 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631ffde-5538-485a-b677-6e20367ae76c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:23.175728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631ffde-5538-485a-b677-6e20367ae76c None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-6631ffde-5538-485a-b677-6e20367ae76c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:30:23.197378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:23.198629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:30:23.212419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:30:23.212419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:23.212419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:23.212419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 473/942] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:23.213703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c17bd-b44f-494b-80d9-944d013fe1a3 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:23.217168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:23.387895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c17bd-b44f-494b-80d9-944d013fe1a3 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:23.441671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713c45d-2009-446e-802d-6b66e9f4b891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:23.442263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713c45d-2009-446e-802d-6b66e9f4b891 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:23.442651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713c45d-2009-446e-802d-6b66e9f4b891 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:23.442923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713c45d-2009-446e-802d-6b66e9f4b891 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:23.443735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713c45d-2009-446e-802d-6b66e9f4b891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:23.444159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713c45d-2009-446e-802d-6b66e9f4b891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:23.445992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713c45d-2009-446e-802d-6b66e9f4b891 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:23.463791 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 470/943] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:23.470757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631ffde-5538-485a-b677-6e20367ae76c None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:23.535957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:23.618003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631ffde-5538-485a-b677-6e20367ae76c None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:23.655099 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d818801-0a45-478f-b585-fb35ee7041e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:23.655734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d818801-0a45-478f-b585-fb35ee7041e6 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:23.656009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d818801-0a45-478f-b585-fb35ee7041e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:23.656281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d818801-0a45-478f-b585-fb35ee7041e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:23.664955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d818801-0a45-478f-b585-fb35ee7041e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:23.666499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d818801-0a45-478f-b585-fb35ee7041e6 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-3d818801-0a45-478f-b585-fb35ee7041e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:30:23.681321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:23.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:23.684973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 474/944] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:30:23.689115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713c45d-2009-446e-802d-6b66e9f4b891 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:23.770680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:23.841249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713c45d-2009-446e-802d-6b66e9f4b891 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:23.901316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5375d38-6d0b-4697-9ec5-1e536f36af11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:23.902199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5375d38-6d0b-4697-9ec5-1e536f36af11 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:23.902600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5375d38-6d0b-4697-9ec5-1e536f36af11 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:23.903114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5375d38-6d0b-4697-9ec5-1e536f36af11 None admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:23.903986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5375d38-6d0b-4697-9ec5-1e536f36af11 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:23.904430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5375d38-6d0b-4697-9ec5-1e536f36af11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:23.906305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5375d38-6d0b-4697-9ec5-1e536f36af11 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:23.930429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 471/945] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Oct 18 11:30:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:30:23.939156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d818801-0a45-478f-b585-fb35ee7041e6 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:24.091292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d818801-0a45-478f-b585-fb35ee7041e6 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:24.093640 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:24.144517 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa11d5-4029-4473-8fac-e5e70a52edd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:24.145671 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa11d5-4029-4473-8fac-e5e70a52edd6 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:24.145987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa11d5-4029-4473-8fac-e5e70a52edd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:24.146397 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa11d5-4029-4473-8fac-e5e70a52edd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:24.163946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa11d5-4029-4473-8fac-e5e70a52edd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:24.165849 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa11d5-4029-4473-8fac-e5e70a52edd6 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-fcfa11d5-4029-4473-8fac-e5e70a52edd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:30:24.190774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:24.192246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:30:24.193572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:30:24.193572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:24.193572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:24.193572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 475/946] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:24.201341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5375d38-6d0b-4697-9ec5-1e536f36af11 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:24.358700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5375d38-6d0b-4697-9ec5-1e536f36af11 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:24.382269 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:24.399898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178632dd-8b0e-46ff-a478-2b74b540a434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:24.400664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178632dd-8b0e-46ff-a478-2b74b540a434 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:24.401170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178632dd-8b0e-46ff-a478-2b74b540a434 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:24.401646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178632dd-8b0e-46ff-a478-2b74b540a434 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:24.402804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178632dd-8b0e-46ff-a478-2b74b540a434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:24.403354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178632dd-8b0e-46ff-a478-2b74b540a434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:24.405878 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178632dd-8b0e-46ff-a478-2b74b540a434 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:24.423013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 472/947] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:24.429495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa11d5-4029-4473-8fac-e5e70a52edd6 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:24.585815 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa11d5-4029-4473-8fac-e5e70a52edd6 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:24.633741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7815039-3ab2-4f35-879e-d6022cc32c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:24.634810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7815039-3ab2-4f35-879e-d6022cc32c4d None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:24.635283 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7815039-3ab2-4f35-879e-d6022cc32c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:24.635755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7815039-3ab2-4f35-879e-d6022cc32c4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:24.650017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7815039-3ab2-4f35-879e-d6022cc32c4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:24.652960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7815039-3ab2-4f35-879e-d6022cc32c4d None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-d7815039-3ab2-4f35-879e-d6022cc32c4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:30:24.675762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:24.677578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:24.679315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:30:24.679315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:30:24.679315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:30:24.679315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:30:24.679315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:30:24.679315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:30:24.679315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:30:24.679315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:24.679315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 476/948] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:30:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:30:24.687794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178632dd-8b0e-46ff-a478-2b74b540a434 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:24.833173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178632dd-8b0e-46ff-a478-2b74b540a434 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:24.876259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175110c-052a-4b4c-ae65-eb57305e5a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:24.877034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175110c-052a-4b4c-ae65-eb57305e5a3d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:24.877314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175110c-052a-4b4c-ae65-eb57305e5a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:24.877583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175110c-052a-4b4c-ae65-eb57305e5a3d None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:24.878293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175110c-052a-4b4c-ae65-eb57305e5a3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:24.878591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175110c-052a-4b4c-ae65-eb57305e5a3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:24.880114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175110c-052a-4b4c-ae65-eb57305e5a3d None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:24.894365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 473/949] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:30:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:24.906017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7815039-3ab2-4f35-879e-d6022cc32c4d None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:24.930971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:25.062601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7815039-3ab2-4f35-879e-d6022cc32c4d None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:25.117789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f99f1-c614-4cfc-b8cf-8ca87fbe03ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:25.118866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f99f1-c614-4cfc-b8cf-8ca87fbe03ac None admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:25.119448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f99f1-c614-4cfc-b8cf-8ca87fbe03ac None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:25.119824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f99f1-c614-4cfc-b8cf-8ca87fbe03ac None admin] PATH_INFO: `/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/99ec4a4276f345c2998723e554ab9608/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:25.122606 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f99f1-c614-4cfc-b8cf-8ca87fbe03ac None admin] RBAC: Authorizing `identity:create_grant(project_id=4daff69874da475b99a7c445df7b3f4d, user_id=99ec4a4276f345c2998723e554ab9608, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:25.124504 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f99f1-c614-4cfc-b8cf-8ca87fbe03ac None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:25.141748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 477/950] 10.222.0.17 () {58 vars in 1439 bytes} [Tue Oct 18 11:30:24 2022] PUT /identity/v3/projects/4daff69874da475b99a7c445df7b3f4d/users/99ec4a4276f345c2998723e554ab9608/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:30:27.586509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd85c6-8c8f-42aa-9726-a433aaa57cd6 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:27.587069 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd85c6-8c8f-42aa-9726-a433aaa57cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:27.587069 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd85c6-8c8f-42aa-9726-a433aaa57cd6 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:27.588334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 474/951] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:30:27.596253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99597b5-9d96-49ea-9681-aa73d1cd9beb None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:27.597133 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99597b5-9d96-49ea-9681-aa73d1cd9beb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:27.597306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99597b5-9d96-49ea-9681-aa73d1cd9beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:27.622762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b99597b5-9d96-49ea-9681-aa73d1cd9beb None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:30:27.624426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b99597b5-9d96-49ea-9681-aa73d1cd9beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:30:27.729892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 478/952] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:30:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:30:28.066271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd85c6-8c8f-42aa-9726-a433aaa57cd6 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:28.142566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:28.249381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd85c6-8c8f-42aa-9726-a433aaa57cd6 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:28.293623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5adbaf-a522-4d2e-90d9-da52c94a62a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:28.294282 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5adbaf-a522-4d2e-90d9-da52c94a62a9 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:28.294643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5adbaf-a522-4d2e-90d9-da52c94a62a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:28.294992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5adbaf-a522-4d2e-90d9-da52c94a62a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:28.307750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5adbaf-a522-4d2e-90d9-da52c94a62a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-1f5adbaf-a522-4d2e-90d9-da52c94a62a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 18 11:30:28.313587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:28.315275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:28.316634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 475/953] 10.222.0.17 () {56 vars in 1096 bytes} [Tue Oct 18 11:30:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:28.322787 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99597b5-9d96-49ea-9681-aa73d1cd9beb None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:28.327586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:28.486250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99597b5-9d96-49ea-9681-aa73d1cd9beb None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:28.534487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16651844-eaec-4008-b813-df3ce41517fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:28.535117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16651844-eaec-4008-b813-df3ce41517fd None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:28.535482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16651844-eaec-4008-b813-df3ce41517fd None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:28.535656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16651844-eaec-4008-b813-df3ce41517fd None admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:28.536328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16651844-eaec-4008-b813-df3ce41517fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:28.536621 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16651844-eaec-4008-b813-df3ce41517fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:28.538102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16651844-eaec-4008-b813-df3ce41517fd None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:28.552077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 479/954] 10.222.0.17 () {56 vars in 1099 bytes} [Tue Oct 18 11:30:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:28.558972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5adbaf-a522-4d2e-90d9-da52c94a62a9 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:28.703761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:28.707680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5adbaf-a522-4d2e-90d9-da52c94a62a9 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:28.765126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78b0f-5b52-423e-a402-310f690cb8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:28.766161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78b0f-5b52-423e-a402-310f690cb8da None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:28.766641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78b0f-5b52-423e-a402-310f690cb8da None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:28.767235 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f78b0f-5b52-423e-a402-310f690cb8da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:28.782015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f78b0f-5b52-423e-a402-310f690cb8da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:28.783787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f78b0f-5b52-423e-a402-310f690cb8da None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-14f78b0f-5b52-423e-a402-310f690cb8da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:30:28.802491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:28.805343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:30:28.807604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:30:28.807604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:28.807604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:28.807604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 476/955] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 18 11:30:28.811812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16651844-eaec-4008-b813-df3ce41517fd None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:28.927474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:28.987373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16651844-eaec-4008-b813-df3ce41517fd None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:29.048515 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33d892-b806-4733-aec8-eacd775abdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:29.049261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33d892-b806-4733-aec8-eacd775abdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:29.049537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33d892-b806-4733-aec8-eacd775abdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:29.049803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33d892-b806-4733-aec8-eacd775abdb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:29.050644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33d892-b806-4733-aec8-eacd775abdb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:29.050996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33d892-b806-4733-aec8-eacd775abdb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:29.052626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33d892-b806-4733-aec8-eacd775abdb3 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:29.071526 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 480/956] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:30:29.078511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78b0f-5b52-423e-a402-310f690cb8da None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:29.267133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f78b0f-5b52-423e-a402-310f690cb8da None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:29.339045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39077efc-9cdc-48f0-9bd3-74bb95ee694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:29.340335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39077efc-9cdc-48f0-9bd3-74bb95ee694d None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:29.340796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39077efc-9cdc-48f0-9bd3-74bb95ee694d None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:29.341263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39077efc-9cdc-48f0-9bd3-74bb95ee694d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:29.357097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39077efc-9cdc-48f0-9bd3-74bb95ee694d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:29.359171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39077efc-9cdc-48f0-9bd3-74bb95ee694d None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-39077efc-9cdc-48f0-9bd3-74bb95ee694d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 18 11:30:29.379720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:29.381400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:29.382715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 477/957] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:29.391379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33d892-b806-4733-aec8-eacd775abdb3 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:29.593750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33d892-b806-4733-aec8-eacd775abdb3 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:29.643941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:29.662077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f39fbf-cd1f-4e90-9bbf-74795cb569a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:29.663734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f39fbf-cd1f-4e90-9bbf-74795cb569a7 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:29.664379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f39fbf-cd1f-4e90-9bbf-74795cb569a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:29.665084 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f39fbf-cd1f-4e90-9bbf-74795cb569a7 None admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:29.666547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f39fbf-cd1f-4e90-9bbf-74795cb569a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:29.667128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f39fbf-cd1f-4e90-9bbf-74795cb569a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:29.670234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f39fbf-cd1f-4e90-9bbf-74795cb569a7 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:29.704915 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 481/958] 10.222.0.17 () {56 vars in 1143 bytes} [Tue Oct 18 11:30:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:29.714671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39077efc-9cdc-48f0-9bd3-74bb95ee694d None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:29.735957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:29.871350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39077efc-9cdc-48f0-9bd3-74bb95ee694d None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:29.923591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fe2d6-1173-40ea-a9d8-01703910ac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:29.924278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fe2d6-1173-40ea-a9d8-01703910ac6a None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:29.924601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fe2d6-1173-40ea-a9d8-01703910ac6a None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:29.924963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fe2d6-1173-40ea-a9d8-01703910ac6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:30.027292 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fe2d6-1173-40ea-a9d8-01703910ac6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:30.029621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fe2d6-1173-40ea-a9d8-01703910ac6a None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:30.056697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:30.201959 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 478/959] 10.222.0.17 () {58 vars in 1282 bytes} [Tue Oct 18 11:30:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 18 11:30:30.208826 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f39fbf-cd1f-4e90-9bbf-74795cb569a7 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:30.369822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f39fbf-cd1f-4e90-9bbf-74795cb569a7 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:30.417881 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bdf88-45d4-4214-8999-47e446049b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:30.418531 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bdf88-45d4-4214-8999-47e446049b05 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:30.418830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bdf88-45d4-4214-8999-47e446049b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:30.419102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bdf88-45d4-4214-8999-47e446049b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:30.432021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bdf88-45d4-4214-8999-47e446049b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:30.433441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bdf88-45d4-4214-8999-47e446049b05 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-f91bdf88-45d4-4214-8999-47e446049b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 18 11:30:30.450553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:30.452279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 18 11:30:30.453974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 18 11:30:30.453974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 18 11:30:30.453974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:30.453974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 482/960] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:30.458117 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fe2d6-1173-40ea-a9d8-01703910ac6a None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:30.579451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fe2d6-1173-40ea-a9d8-01703910ac6a None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:30.595082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:30.623803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ada086-3db2-4eb7-8412-5d8d133860b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:30.624493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ada086-3db2-4eb7-8412-5d8d133860b6 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:30.624771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ada086-3db2-4eb7-8412-5d8d133860b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:30.625070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ada086-3db2-4eb7-8412-5d8d133860b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:30.626074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ada086-3db2-4eb7-8412-5d8d133860b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:30.626594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ada086-3db2-4eb7-8412-5d8d133860b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:30.628185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ada086-3db2-4eb7-8412-5d8d133860b6 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:30.650541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 479/961] 10.222.0.17 () {56 vars in 1109 bytes} [Tue Oct 18 11:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:30.652411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bdf88-45d4-4214-8999-47e446049b05 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:30.698154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:30.792401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bdf88-45d4-4214-8999-47e446049b05 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:30.852256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5573525-9aa6-4350-886b-8478ba4e3d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:30.852998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5573525-9aa6-4350-886b-8478ba4e3d1e None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:30.853279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5573525-9aa6-4350-886b-8478ba4e3d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:30.853590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5573525-9aa6-4350-886b-8478ba4e3d1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:30.863080 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5573525-9aa6-4350-886b-8478ba4e3d1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:30.864595 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5573525-9aa6-4350-886b-8478ba4e3d1e None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application [None req-e5573525-9aa6-4350-886b-8478ba4e3d1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 18 11:30:30.878788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:30.879918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:30.881794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 18 11:30:30.881794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 18 11:30:30.881794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 18 11:30:30.881794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 18 11:30:30.881794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 18 11:30:30.881794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 18 11:30:30.881794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 18 11:30:30.881794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: ERROR keystone.server.flask.application Oct 18 11:30:30.883835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 483/962] 10.222.0.17 () {56 vars in 1111 bytes} [Tue Oct 18 11:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:30.890378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ada086-3db2-4eb7-8412-5d8d133860b6 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:31.039331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ada086-3db2-4eb7-8412-5d8d133860b6 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:31.085917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ecad29-8413-403f-b2b6-e22e4c44e7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:31.086545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecad29-8413-403f-b2b6-e22e4c44e7eb None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:31.086819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecad29-8413-403f-b2b6-e22e4c44e7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:31.087084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecad29-8413-403f-b2b6-e22e4c44e7eb None admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:31.087796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ecad29-8413-403f-b2b6-e22e4c44e7eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:31.088101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ecad29-8413-403f-b2b6-e22e4c44e7eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:31.090963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ecad29-8413-403f-b2b6-e22e4c44e7eb None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:31.104067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 480/963] 10.222.0.17 () {56 vars in 1148 bytes} [Tue Oct 18 11:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:30:31.111966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5573525-9aa6-4350-886b-8478ba4e3d1e None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:31.255832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:31.264827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5573525-9aa6-4350-886b-8478ba4e3d1e None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:31.322914 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578eefa3-1468-437b-8c0d-82d052baf1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:31.324202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eefa3-1468-437b-8c0d-82d052baf1ad None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:31.324664 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eefa3-1468-437b-8c0d-82d052baf1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:31.325250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578eefa3-1468-437b-8c0d-82d052baf1ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:31.326756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578eefa3-1468-437b-8c0d-82d052baf1ad None admin] RBAC: Adding query filter params (role.id=8d9ff9da6ad946a6bc49336777182ff9, scope.project.id=4daff69874da475b99a7c445df7b3f4d, user.id=99ec4a4276f345c2998723e554ab9608) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:31.327626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578eefa3-1468-437b-8c0d-82d052baf1ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:31.331073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578eefa3-1468-437b-8c0d-82d052baf1ad None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:31.342167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 484/964] 10.222.0.17 () {56 vars in 1374 bytes} [Tue Oct 18 11:30:31 2022] GET /identity/v3/role_assignments?user.id=99ec4a4276f345c2998723e554ab9608&scope.project.id=4daff69874da475b99a7c445df7b3f4d&role.id=8d9ff9da6ad946a6bc49336777182ff9 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:33.696137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776b42b-92dd-4c7a-a40a-bfa45404d175 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:33.696962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776b42b-92dd-4c7a-a40a-bfa45404d175 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:33.696962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776b42b-92dd-4c7a-a40a-bfa45404d175 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:33.698044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 481/965] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:30:33.703721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8ea8b-412b-4821-a53d-558ad547161a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:33.704261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8ea8b-412b-4821-a53d-558ad547161a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:33.704261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8ea8b-412b-4821-a53d-558ad547161a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:33.726751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3aa8ea8b-412b-4821-a53d-558ad547161a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:30:33.728637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3aa8ea8b-412b-4821-a53d-558ad547161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:30:33.820737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 485/966] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:30:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:30:34.096771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776b42b-92dd-4c7a-a40a-bfa45404d175 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:34.134369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:34.217821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776b42b-92dd-4c7a-a40a-bfa45404d175 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:34.261859 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae3313-c040-4974-88cf-d4019c3d0d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:34.262507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae3313-c040-4974-88cf-d4019c3d0d01 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:34.262784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae3313-c040-4974-88cf-d4019c3d0d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:34.263057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae3313-c040-4974-88cf-d4019c3d0d01 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:34.263659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae3313-c040-4974-88cf-d4019c3d0d01 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:34.265068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae3313-c040-4974-88cf-d4019c3d0d01 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-5cae3313-c040-4974-88cf-d4019c3d0d01 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:34.276159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:34.278368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:34.280491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 482/967] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Oct 18 11:30:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:34.286615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8ea8b-412b-4821-a53d-558ad547161a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:34.394961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:34.406237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8ea8b-412b-4821-a53d-558ad547161a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:34.442835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0585329-1afd-46ba-a7ff-1268c68f34a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:34.443991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0585329-1afd-46ba-a7ff-1268c68f34a5 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:34.444264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0585329-1afd-46ba-a7ff-1268c68f34a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:34.444530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0585329-1afd-46ba-a7ff-1268c68f34a5 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:34.445192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0585329-1afd-46ba-a7ff-1268c68f34a5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:34.445489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0585329-1afd-46ba-a7ff-1268c68f34a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:34.446849 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0585329-1afd-46ba-a7ff-1268c68f34a5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:34.452417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 486/968] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Oct 18 11:30:34 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:34.459408 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae3313-c040-4974-88cf-d4019c3d0d01 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:34.571026 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:34.578787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae3313-c040-4974-88cf-d4019c3d0d01 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:34.616541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3337c-0934-4b9a-822d-034a16cbd0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:34.617198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3337c-0934-4b9a-822d-034a16cbd0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:34.617478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3337c-0934-4b9a-822d-034a16cbd0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:34.617755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3337c-0934-4b9a-822d-034a16cbd0b1 None admin] PATH_INFO: `/v3/services` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:34.618397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3337c-0934-4b9a-822d-034a16cbd0b1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:34.618700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3337c-0934-4b9a-822d-034a16cbd0b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:34.620081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3337c-0934-4b9a-822d-034a16cbd0b1 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:34.625305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 483/969] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Oct 18 11:30:34 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:30:36.864180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e48654-84d1-47ba-a6ff-817168c6e3a9 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:36.864884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e48654-84d1-47ba-a6ff-817168c6e3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:36.865848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e48654-84d1-47ba-a6ff-817168c6e3a9 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:36.867308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 487/970] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:30:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:30:36.874771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfbdef-0b93-4663-a7a2-3fa63d4439c6 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:36.875509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfbdef-0b93-4663-a7a2-3fa63d4439c6 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:36.875695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfbdef-0b93-4663-a7a2-3fa63d4439c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:36.909396 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-44bfbdef-0b93-4663-a7a2-3fa63d4439c6 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:30:36.911607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-44bfbdef-0b93-4663-a7a2-3fa63d4439c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:30:37.006244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 484/971] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:30:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 18 11:30:37.291978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e48654-84d1-47ba-a6ff-817168c6e3a9 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:37.441613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e48654-84d1-47ba-a6ff-817168c6e3a9 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:37.508047 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e02682-89eb-4c52-a71c-0fbac7706cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:37.509156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e02682-89eb-4c52-a71c-0fbac7706cc5 None admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:37.509663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e02682-89eb-4c52-a71c-0fbac7706cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:37.510162 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e02682-89eb-4c52-a71c-0fbac7706cc5 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:37.511408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e02682-89eb-4c52-a71c-0fbac7706cc5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:37.514061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e02682-89eb-4c52-a71c-0fbac7706cc5 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:37.536892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 488/972] 10.222.0.17 () {60 vars in 1141 bytes} [Tue Oct 18 11:30:37 2022] POST /identity/v3/services => generated 246 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:30:39.847304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25a763-e065-4786-8751-73fe6040ed39 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:39.847951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25a763-e065-4786-8751-73fe6040ed39 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:39.847951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25a763-e065-4786-8751-73fe6040ed39 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:39.850054 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 485/973] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:30:39.857779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed534ea-bc4e-410d-b335-abb1aa1d3d6e None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:39.858375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed534ea-bc4e-410d-b335-abb1aa1d3d6e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:39.859031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed534ea-bc4e-410d-b335-abb1aa1d3d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:39.888561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-bed534ea-bc4e-410d-b335-abb1aa1d3d6e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:30:39.890311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-bed534ea-bc4e-410d-b335-abb1aa1d3d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:30:39.908666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:40.000444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 489/974] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:30:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:30:40.293424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25a763-e065-4786-8751-73fe6040ed39 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:40.427359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:40.441120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25a763-e065-4786-8751-73fe6040ed39 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:40.477039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c099436-28ab-483f-847c-8e862c2522b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:40.477677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c099436-28ab-483f-847c-8e862c2522b7 None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:40.477953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c099436-28ab-483f-847c-8e862c2522b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:40.478221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c099436-28ab-483f-847c-8e862c2522b7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:40.478804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c099436-28ab-483f-847c-8e862c2522b7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:40.480164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c099436-28ab-483f-847c-8e862c2522b7 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-7c099436-28ab-483f-847c-8e862c2522b7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:40.489522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:40.491391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:40.493086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 486/975] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Oct 18 11:30:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:40.497104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed534ea-bc4e-410d-b335-abb1aa1d3d6e None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:40.601166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed534ea-bc4e-410d-b335-abb1aa1d3d6e None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:40.638074 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcc25b-74be-4248-836b-d811f2422e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:40.638701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcc25b-74be-4248-836b-d811f2422e83 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:40.638980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcc25b-74be-4248-836b-d811f2422e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:40.639254 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcc25b-74be-4248-836b-d811f2422e83 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:40.639898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcc25b-74be-4248-836b-d811f2422e83 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:40.640201 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcc25b-74be-4248-836b-d811f2422e83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:40.641616 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdcc25b-74be-4248-836b-d811f2422e83 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:40.648080 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 490/976] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Oct 18 11:30:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:30:40.657799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c099436-28ab-483f-847c-8e862c2522b7 None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:40.826697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c099436-28ab-483f-847c-8e862c2522b7 None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:40.863351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1e533-7301-40a8-9ed0-2c5a55e2f01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:40.864021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1e533-7301-40a8-9ed0-2c5a55e2f01b None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:40.864257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1e533-7301-40a8-9ed0-2c5a55e2f01b None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:40.864511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1e533-7301-40a8-9ed0-2c5a55e2f01b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:40.865849 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1e533-7301-40a8-9ed0-2c5a55e2f01b None admin] RBAC: Adding query filter params (interface=public, service_id=2a8032fe099b46598f0c88f4ead3ada2, region_id=RegionOne) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:40.866160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1e533-7301-40a8-9ed0-2c5a55e2f01b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:40.867537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1e533-7301-40a8-9ed0-2c5a55e2f01b None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:40.873370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 487/977] 10.222.0.17 () {56 vars in 1251 bytes} [Tue Oct 18 11:30:40 2022] GET /identity/v3/endpoints?service_id=2a8032fe099b46598f0c88f4ead3ada2&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:40.883417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcc25b-74be-4248-836b-d811f2422e83 None admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:40.889421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:41.004688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcc25b-74be-4248-836b-d811f2422e83 None admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:41.044328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c915a71-44c8-45a3-9e8a-15a90e15f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:41.044986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c915a71-44c8-45a3-9e8a-15a90e15f627 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:41.045263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c915a71-44c8-45a3-9e8a-15a90e15f627 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:41.045534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c915a71-44c8-45a3-9e8a-15a90e15f627 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:41.046147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c915a71-44c8-45a3-9e8a-15a90e15f627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:41.047548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c915a71-44c8-45a3-9e8a-15a90e15f627 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:41.055916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 491/978] 10.222.0.17 () {56 vars in 1087 bytes} [Tue Oct 18 11:30:40 2022] GET /identity/v3/services => generated 2191 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:30:43.328174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a171b-d26b-48cd-a929-8765cf43d915 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:43.328710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a171b-d26b-48cd-a929-8765cf43d915 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:43.328710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a171b-d26b-48cd-a929-8765cf43d915 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:43.330059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 488/979] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:30:43.336354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a048c5-924a-4a71-9714-45d15c855faf None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:43.336838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a048c5-924a-4a71-9714-45d15c855faf None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:43.336996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a048c5-924a-4a71-9714-45d15c855faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:43.355234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-20a048c5-924a-4a71-9714-45d15c855faf None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:30:43.356442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-20a048c5-924a-4a71-9714-45d15c855faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:30:43.432713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:43.439658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 492/980] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:30:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 18 11:30:43.714543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a171b-d26b-48cd-a929-8765cf43d915 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:43.735473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:43.828324 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a171b-d26b-48cd-a929-8765cf43d915 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:43.876883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91d7ad-6096-4f7d-b7b1-1a30e837fadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:43.878472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91d7ad-6096-4f7d-b7b1-1a30e837fadf None admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:43.878993 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91d7ad-6096-4f7d-b7b1-1a30e837fadf None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:43.879534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91d7ad-6096-4f7d-b7b1-1a30e837fadf None admin] PATH_INFO: `/v3/services/placement` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:43.881571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91d7ad-6096-4f7d-b7b1-1a30e837fadf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:43.884656 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91d7ad-6096-4f7d-b7b1-1a30e837fadf None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application [None req-5c91d7ad-6096-4f7d-b7b1-1a30e837fadf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 18 11:30:43.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 18 11:30:43.907089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.get_or_create( Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application with Lock( Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self._enter() Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application return self.creator() Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application created_value = creator( Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: ERROR keystone.server.flask.application Oct 18 11:30:43.908104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 489/981] 10.222.0.17 () {56 vars in 1117 bytes} [Tue Oct 18 11:30:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 18 11:30:43.916592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a048c5-924a-4a71-9714-45d15c855faf None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:44.028064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a048c5-924a-4a71-9714-45d15c855faf None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:44.068241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a0191-b085-4c66-bf28-bc8cebb28811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:44.068900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a0191-b085-4c66-bf28-bc8cebb28811 None admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:44.069180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a0191-b085-4c66-bf28-bc8cebb28811 None admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:44.069448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a0191-b085-4c66-bf28-bc8cebb28811 None admin] PATH_INFO: `/v3/services` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:44.070092 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a0191-b085-4c66-bf28-bc8cebb28811 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:30:44.070395 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a0191-b085-4c66-bf28-bc8cebb28811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:44.071806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a0191-b085-4c66-bf28-bc8cebb28811 None admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:44.077858 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 493/982] 10.222.0.17 () {56 vars in 1116 bytes} [Tue Oct 18 11:30:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:30:44.087722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91d7ad-6096-4f7d-b7b1-1a30e837fadf None admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:30:44.181827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:44.201862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91d7ad-6096-4f7d-b7b1-1a30e837fadf None admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:30:44.239070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efd24f-1a8d-488a-bb63-64b0be16cce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:30:44.239679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efd24f-1a8d-488a-bb63-64b0be16cce9 None admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:30:44.239964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efd24f-1a8d-488a-bb63-64b0be16cce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:30:44.240236 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efd24f-1a8d-488a-bb63-64b0be16cce9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:30:44.240961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efd24f-1a8d-488a-bb63-64b0be16cce9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:30:44.242333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efd24f-1a8d-488a-bb63-64b0be16cce9 None admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:30:44.263524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 490/983] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:30:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:30:54.075178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=69 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:30:54.258875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:31:57.430959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2f32-e1a2-4935-833e-4b204a37638a None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:31:57.431554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2f32-e1a2-4935-833e-4b204a37638a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:31:57.431554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe2f32-e1a2-4935-833e-4b204a37638a None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:31:57.439132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade6ad5-8e7f-4bbd-9f8a-bed656cb7d64 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:31:57.439882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade6ad5-8e7f-4bbd-9f8a-bed656cb7d64 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:31:57.439882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade6ad5-8e7f-4bbd-9f8a-bed656cb7d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:31:57.444429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 494/984] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:31:57 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:31:57.539389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-aade6ad5-8e7f-4bbd-9f8a-bed656cb7d64 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:31:57.540537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-aade6ad5-8e7f-4bbd-9f8a-bed656cb7d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:31:57.662577 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 491/985] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:31:57.972625 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fac24-3c29-47fa-b8fb-0c02600c7d88 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:31:57.972901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fac24-3c29-47fa-b8fb-0c02600c7d88 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:31:57.973167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fac24-3c29-47fa-b8fb-0c02600c7d88 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:31:57.974747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 495/986] 10.222.0.17 () {54 vars in 964 bytes} [Tue Oct 18 11:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:31:57.986151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed126ead-7c7a-4dc1-877f-80b258473901 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:31:57.986574 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed126ead-7c7a-4dc1-877f-80b258473901 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:31:57.986713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed126ead-7c7a-4dc1-877f-80b258473901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:31:58.205079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-ed126ead-7c7a-4dc1-877f-80b258473901 None None] MFA Rules not processed for user `d301f3d9aeb644a38c31dbdbef2a48c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:31:58.207323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ed126ead-7c7a-4dc1-877f-80b258473901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:31:58.281928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:31:58.435660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 492/987] 10.222.0.17 () {58 vars in 1061 bytes} [Tue Oct 18 11:31:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:31:58.450227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fac24-3c29-47fa-b8fb-0c02600c7d88 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:31:58.452168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-be2fac24-3c29-47fa-b8fb-0c02600c7d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:31:58.614669 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fac24-3c29-47fa-b8fb-0c02600c7d88 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:31:58.708536 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee02c37-31df-49e3-9b6f-2fe8d851ceea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:31:58.709066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee02c37-31df-49e3-9b6f-2fe8d851ceea service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:31:58.709347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee02c37-31df-49e3-9b6f-2fe8d851ceea service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:31:58.709637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee02c37-31df-49e3-9b6f-2fe8d851ceea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:31:58.711402 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8ee02c37-31df-49e3-9b6f-2fe8d851ceea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:31:58.779517 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:31:58.852820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee02c37-31df-49e3-9b6f-2fe8d851ceea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:31:58.854354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee02c37-31df-49e3-9b6f-2fe8d851ceea service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:31:58.877815 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 496/988] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:01.694484 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55f5c9-2b34-4f8f-9b28-14d1d0710e1a None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:01.695009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55f5c9-2b34-4f8f-9b28-14d1d0710e1a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:01.695009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55f5c9-2b34-4f8f-9b28-14d1d0710e1a None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:01.696387 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 493/989] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:01.702149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647fad1-6639-420f-885a-d68bfec609a9 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:01.702684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647fad1-6639-420f-885a-d68bfec609a9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:01.702802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647fad1-6639-420f-885a-d68bfec609a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:01.778288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-5647fad1-6639-420f-885a-d68bfec609a9 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:01.779531 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5647fad1-6639-420f-885a-d68bfec609a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:01.882321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 497/990] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:02.183567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b089a1-5d8a-4d7a-9f8b-10ba293702be None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:02.183903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b089a1-5d8a-4d7a-9f8b-10ba293702be None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:02.184225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b089a1-5d8a-4d7a-9f8b-10ba293702be None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:02.185731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 494/991] 10.222.0.17 () {54 vars in 964 bytes} [Tue Oct 18 11:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:02.192696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05245f5e-c1a1-4020-932a-5274810d5c7e None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:02.193107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05245f5e-c1a1-4020-932a-5274810d5c7e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:02.193398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05245f5e-c1a1-4020-932a-5274810d5c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:02.206911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:02.277685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-05245f5e-c1a1-4020-932a-5274810d5c7e None None] MFA Rules not processed for user `d301f3d9aeb644a38c31dbdbef2a48c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:02.278922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-05245f5e-c1a1-4020-932a-5274810d5c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:02.403811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 498/992] 10.222.0.17 () {58 vars in 1061 bytes} [Tue Oct 18 11:32:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:02.413991 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b089a1-5d8a-4d7a-9f8b-10ba293702be None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:02.415503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-86b089a1-5d8a-4d7a-9f8b-10ba293702be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:02.526489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b089a1-5d8a-4d7a-9f8b-10ba293702be None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:02.644211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:02.647141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10d3ac-d45a-45c7-8834-555615b6c119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:02.648025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10d3ac-d45a-45c7-8834-555615b6c119 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:02.648576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10d3ac-d45a-45c7-8834-555615b6c119 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:02.649053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10d3ac-d45a-45c7-8834-555615b6c119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:02.651054 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6e10d3ac-d45a-45c7-8834-555615b6c119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:02.769062 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10d3ac-d45a-45c7-8834-555615b6c119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:02.770485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10d3ac-d45a-45c7-8834-555615b6c119 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:02.791010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 495/993] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:05.659142 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cbd33-513d-470e-a956-fad7cec8ec6e None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:05.659741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cbd33-513d-470e-a956-fad7cec8ec6e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:05.659741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cbd33-513d-470e-a956-fad7cec8ec6e None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:05.661118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 499/994] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:32:05.667472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4c473-7747-4529-ae6e-9e7aed46dac3 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:05.668221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4c473-7747-4529-ae6e-9e7aed46dac3 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:05.668221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4c473-7747-4529-ae6e-9e7aed46dac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:05.742195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-33e4c473-7747-4529-ae6e-9e7aed46dac3 None None] MFA Rules not processed for user `e9c3b13d60ab475ba56f29d0a183c8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:05.743562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-33e4c473-7747-4529-ae6e-9e7aed46dac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:05.801257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:05.859057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 496/995] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:05 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:32:06.161216 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cbd33-513d-470e-a956-fad7cec8ec6e None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:06.162898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-344cbd33-513d-470e-a956-fad7cec8ec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:06.277309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cbd33-513d-470e-a956-fad7cec8ec6e None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:06.280510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:06.379038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec420298-9718-4d88-8786-c57dab67cb69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:06.379660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec420298-9718-4d88-8786-c57dab67cb69 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:06.379942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec420298-9718-4d88-8786-c57dab67cb69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:06.380215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec420298-9718-4d88-8786-c57dab67cb69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:06.381884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ec420298-9718-4d88-8786-c57dab67cb69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:06.497059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec420298-9718-4d88-8786-c57dab67cb69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:06.498403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec420298-9718-4d88-8786-c57dab67cb69 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:06.518188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 500/996] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:06 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:09.508453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b0889-022b-485d-8a78-fe807d8123d6 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:09.509281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b0889-022b-485d-8a78-fe807d8123d6 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:09.509281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b0889-022b-485d-8a78-fe807d8123d6 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:09.510258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 497/997] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:32:09.515873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd80a5-ff85-46bd-a1ce-9a6979cf5177 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:09.516594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd80a5-ff85-46bd-a1ce-9a6979cf5177 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:09.516594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd80a5-ff85-46bd-a1ce-9a6979cf5177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:09.562190 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:09.592696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a8fd80a5-ff85-46bd-a1ce-9a6979cf5177 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:09.593974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a8fd80a5-ff85-46bd-a1ce-9a6979cf5177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:09.704184 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 501/998] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:32:09.997228 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b0889-022b-485d-8a78-fe807d8123d6 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:09.998782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d95b0889-022b-485d-8a78-fe807d8123d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:10.103695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b0889-022b-485d-8a78-fe807d8123d6 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:10.189119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925b44b-926b-4fd3-95c9-b63317819fd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:10.189725 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925b44b-926b-4fd3-95c9-b63317819fd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:10.189997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925b44b-926b-4fd3-95c9-b63317819fd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:10.190263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925b44b-926b-4fd3-95c9-b63317819fd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:10.191865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a925b44b-926b-4fd3-95c9-b63317819fd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:10.199600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:10.307658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925b44b-926b-4fd3-95c9-b63317819fd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:10.309025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925b44b-926b-4fd3-95c9-b63317819fd3 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:10.327755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 498/999] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:13.241098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48df39-7009-4f71-bcc1-269837340219 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:13.241860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48df39-7009-4f71-bcc1-269837340219 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:13.241860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48df39-7009-4f71-bcc1-269837340219 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:13.242971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 502/1000] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:13.250327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6a872-2cbe-4010-b0b7-33842b301bc4 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:13.250863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6a872-2cbe-4010-b0b7-33842b301bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:13.251322 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6a872-2cbe-4010-b0b7-33842b301bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:13.325381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-b2e6a872-2cbe-4010-b0b7-33842b301bc4 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:13.326638 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b2e6a872-2cbe-4010-b0b7-33842b301bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:13.396191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:13.444964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 499/1001] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:13.730967 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48df39-7009-4f71-bcc1-269837340219 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:13.732692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3c48df39-7009-4f71-bcc1-269837340219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:13.858011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48df39-7009-4f71-bcc1-269837340219 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:13.871649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:13.946506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb67b2e-99aa-4b6d-b264-a2734b414e1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:13.947046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb67b2e-99aa-4b6d-b264-a2734b414e1e service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:13.947339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb67b2e-99aa-4b6d-b264-a2734b414e1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:13.947607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb67b2e-99aa-4b6d-b264-a2734b414e1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:13.949275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2bb67b2e-99aa-4b6d-b264-a2734b414e1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:14.070147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb67b2e-99aa-4b6d-b264-a2734b414e1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:14.071467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb67b2e-99aa-4b6d-b264-a2734b414e1e service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:14.091226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 503/1002] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:16.459476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c623d17-a3e4-4454-9a7f-7af35e95557e None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:16.460227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c623d17-a3e4-4454-9a7f-7af35e95557e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:16.460227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c623d17-a3e4-4454-9a7f-7af35e95557e None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:16.461394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 500/1003] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:16.468468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85bb0a-bd22-4ec0-b490-5bf5a1f17547 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:16.469097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85bb0a-bd22-4ec0-b490-5bf5a1f17547 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:16.469481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85bb0a-bd22-4ec0-b490-5bf5a1f17547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:16.536734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:16.565708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-be85bb0a-bd22-4ec0-b490-5bf5a1f17547 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:16.566922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-be85bb0a-bd22-4ec0-b490-5bf5a1f17547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:16.677677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 504/1004] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:16.961765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c623d17-a3e4-4454-9a7f-7af35e95557e None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:16.963653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7c623d17-a3e4-4454-9a7f-7af35e95557e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:17.075201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c623d17-a3e4-4454-9a7f-7af35e95557e None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:17.161703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a794d7-b1db-4282-8475-2756fc55e195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:17.162319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a794d7-b1db-4282-8475-2756fc55e195 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:17.162599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a794d7-b1db-4282-8475-2756fc55e195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:17.162872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a794d7-b1db-4282-8475-2756fc55e195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:17.164532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f9a794d7-b1db-4282-8475-2756fc55e195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:17.184319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:17.298802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a794d7-b1db-4282-8475-2756fc55e195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:17.300771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a794d7-b1db-4282-8475-2756fc55e195 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:17.335772 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 501/1005] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:19.610545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d6c88-ff84-4c14-8934-eb994aee7d08 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:19.611042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d6c88-ff84-4c14-8934-eb994aee7d08 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:19.611042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d6c88-ff84-4c14-8934-eb994aee7d08 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:19.612351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 505/1006] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:19.618714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362513-82c6-4d74-b387-3594471443e3 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:19.618714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362513-82c6-4d74-b387-3594471443e3 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:19.618714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362513-82c6-4d74-b387-3594471443e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:19.693274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-53362513-82c6-4d74-b387-3594471443e3 None None] MFA Rules not processed for user `e9c3b13d60ab475ba56f29d0a183c8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:19.694477 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-53362513-82c6-4d74-b387-3594471443e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:19.763341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:19.804634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 502/1007] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:19 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:20.087802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d6c88-ff84-4c14-8934-eb994aee7d08 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:20.089431 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5e0d6c88-ff84-4c14-8934-eb994aee7d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:20.206085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d6c88-ff84-4c14-8934-eb994aee7d08 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:20.228076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:20.287157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b20ff-bc90-43cc-b877-d0e168a5fff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:20.287766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b20ff-bc90-43cc-b877-d0e168a5fff8 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:20.288039 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b20ff-bc90-43cc-b877-d0e168a5fff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:20.288301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b20ff-bc90-43cc-b877-d0e168a5fff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:20.289942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ef1b20ff-bc90-43cc-b877-d0e168a5fff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:20.398970 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b20ff-bc90-43cc-b877-d0e168a5fff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:20.400429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b20ff-bc90-43cc-b877-d0e168a5fff8 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:20.418359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 506/1008] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:22.878416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391e961-bf73-4645-acf7-57983b5f5efd None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:22.878987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391e961-bf73-4645-acf7-57983b5f5efd None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:22.878987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391e961-bf73-4645-acf7-57983b5f5efd None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:22.880291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 503/1009] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:32:22.885503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a3940-7c1b-4d45-ae6d-b557c3271bc6 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:22.885907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a3940-7c1b-4d45-ae6d-b557c3271bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:22.886052 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a3940-7c1b-4d45-ae6d-b557c3271bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:22.949137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:22.952424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-784a3940-7c1b-4d45-ae6d-b557c3271bc6 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:22.954533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-784a3940-7c1b-4d45-ae6d-b557c3271bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:23.049360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 507/1010] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:23.327511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391e961-bf73-4645-acf7-57983b5f5efd None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:23.329222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6391e961-bf73-4645-acf7-57983b5f5efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:23.432211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391e961-bf73-4645-acf7-57983b5f5efd None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:23.504741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e1a8c-2f7d-4c33-b6a0-2449c999f047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:23.505366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e1a8c-2f7d-4c33-b6a0-2449c999f047 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:23.505643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e1a8c-2f7d-4c33-b6a0-2449c999f047 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:23.505905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e1a8c-2f7d-4c33-b6a0-2449c999f047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:23.507482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3a8e1a8c-2f7d-4c33-b6a0-2449c999f047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:23.538232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:23.618979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e1a8c-2f7d-4c33-b6a0-2449c999f047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:23.620419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e1a8c-2f7d-4c33-b6a0-2449c999f047 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:23.637950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 504/1011] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:25.857663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c7929-9536-4b68-93c7-7044b018f135 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:25.858233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c7929-9536-4b68-93c7-7044b018f135 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:25.858233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c7929-9536-4b68-93c7-7044b018f135 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:25.859585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 508/1012] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:25.864681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eaee9-cb18-4ab6-9ece-c6e11dfc2928 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:25.865110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eaee9-cb18-4ab6-9ece-c6e11dfc2928 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:25.865271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eaee9-cb18-4ab6-9ece-c6e11dfc2928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:25.929529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-7c8eaee9-cb18-4ab6-9ece-c6e11dfc2928 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:25.930726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7c8eaee9-cb18-4ab6-9ece-c6e11dfc2928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:26.003573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:26.026218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 505/1013] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:26.310363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c7929-9536-4b68-93c7-7044b018f135 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:26.312438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4e0c7929-9536-4b68-93c7-7044b018f135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:26.415669 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c7929-9536-4b68-93c7-7044b018f135 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:26.444794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:26.490496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05272604-53b0-47e7-bf4c-a9657fa1883e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:26.491114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05272604-53b0-47e7-bf4c-a9657fa1883e service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:26.491392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05272604-53b0-47e7-bf4c-a9657fa1883e service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:26.491664 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05272604-53b0-47e7-bf4c-a9657fa1883e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:26.493331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-05272604-53b0-47e7-bf4c-a9657fa1883e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:26.607585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05272604-53b0-47e7-bf4c-a9657fa1883e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:26.609083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05272604-53b0-47e7-bf4c-a9657fa1883e service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:26.626970 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 509/1014] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:29.108270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4d664-a0ea-42e1-9bae-2dee681894eb None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:29.109483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4d664-a0ea-42e1-9bae-2dee681894eb None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:29.109483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4d664-a0ea-42e1-9bae-2dee681894eb None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:29.110791 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 506/1015] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:29.115937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1da40-9902-4874-91f5-7949fc3e20cc None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:29.116355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1da40-9902-4874-91f5-7949fc3e20cc None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:29.116481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1da40-9902-4874-91f5-7949fc3e20cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:29.185492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-5cf1da40-9902-4874-91f5-7949fc3e20cc None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:29.186698 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5cf1da40-9902-4874-91f5-7949fc3e20cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:29.195225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:29.282321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 510/1016] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:29.566839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4d664-a0ea-42e1-9bae-2dee681894eb None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:29.568391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3ef4d664-a0ea-42e1-9bae-2dee681894eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:29.672001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4d664-a0ea-42e1-9bae-2dee681894eb None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:29.748229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b488176-fcb6-4101-957b-3397e4a8d7d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:29.748833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b488176-fcb6-4101-957b-3397e4a8d7d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:29.749133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b488176-fcb6-4101-957b-3397e4a8d7d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:29.749966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b488176-fcb6-4101-957b-3397e4a8d7d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:29.751540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9b488176-fcb6-4101-957b-3397e4a8d7d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:29.788587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:29.862322 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b488176-fcb6-4101-957b-3397e4a8d7d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:29.863651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b488176-fcb6-4101-957b-3397e4a8d7d7 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:29.881290 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 507/1017] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:34.033120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f142bd-62a5-418d-a028-99b7557b6ecb None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:34.033661 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f142bd-62a5-418d-a028-99b7557b6ecb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:34.033661 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f142bd-62a5-418d-a028-99b7557b6ecb None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:34.035065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 511/1018] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:34.041130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a04d8-3794-4e06-9a4c-80698be60512 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:34.041682 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a04d8-3794-4e06-9a4c-80698be60512 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:34.041682 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a04d8-3794-4e06-9a4c-80698be60512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:34.116373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-289a04d8-3794-4e06-9a4c-80698be60512 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:34.117668 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-289a04d8-3794-4e06-9a4c-80698be60512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:34.234677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:34.250042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 508/1019] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:34.537871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f142bd-62a5-418d-a028-99b7557b6ecb None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:34.539403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c7f142bd-62a5-418d-a028-99b7557b6ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:34.646969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f142bd-62a5-418d-a028-99b7557b6ecb None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:34.685988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:34.723596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3decd-bf3e-4659-bd04-4b01e60c7e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:34.724229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3decd-bf3e-4659-bd04-4b01e60c7e9f service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:34.724501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3decd-bf3e-4659-bd04-4b01e60c7e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:34.724766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3decd-bf3e-4659-bd04-4b01e60c7e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:34.726483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-87c3decd-bf3e-4659-bd04-4b01e60c7e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:34.839368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3decd-bf3e-4659-bd04-4b01e60c7e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:34.840711 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3decd-bf3e-4659-bd04-4b01e60c7e9f service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:34.861264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 512/1020] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:37.699792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa93717-f0b1-4fbe-a7ec-f148e13e99a7 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:37.700363 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa93717-f0b1-4fbe-a7ec-f148e13e99a7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:37.700456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa93717-f0b1-4fbe-a7ec-f148e13e99a7 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:37.701781 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 509/1021] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:37.707107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461cdaf-a7c1-4eaf-a99e-e8bda42ac697 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:37.707540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461cdaf-a7c1-4eaf-a99e-e8bda42ac697 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:37.707649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461cdaf-a7c1-4eaf-a99e-e8bda42ac697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:37.775710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e461cdaf-a7c1-4eaf-a99e-e8bda42ac697 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:37.777174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e461cdaf-a7c1-4eaf-a99e-e8bda42ac697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:37.796341 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:37.886887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 513/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:38.173481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa93717-f0b1-4fbe-a7ec-f148e13e99a7 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:38.175036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4aa93717-f0b1-4fbe-a7ec-f148e13e99a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:38.286097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa93717-f0b1-4fbe-a7ec-f148e13e99a7 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:38.359969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af5769-099f-4950-8f5d-0f92ca6e59d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:38.360497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af5769-099f-4950-8f5d-0f92ca6e59d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:38.360777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af5769-099f-4950-8f5d-0f92ca6e59d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:38.361082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af5769-099f-4950-8f5d-0f92ca6e59d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:38.362675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-96af5769-099f-4950-8f5d-0f92ca6e59d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:38.406495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:38.475470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af5769-099f-4950-8f5d-0f92ca6e59d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:38.476943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af5769-099f-4950-8f5d-0f92ca6e59d2 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:38.495893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 510/1023] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:43.097721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7177f13-49d9-490d-96ec-c4cfd7b77cf3 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:43.098277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7177f13-49d9-490d-96ec-c4cfd7b77cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:43.098277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7177f13-49d9-490d-96ec-c4cfd7b77cf3 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:43.099600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 514/1024] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:43.105532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681b6a3-cd96-4edc-b14d-8548a1f78582 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:43.105967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681b6a3-cd96-4edc-b14d-8548a1f78582 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:43.106143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681b6a3-cd96-4edc-b14d-8548a1f78582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:43.183115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c681b6a3-cd96-4edc-b14d-8548a1f78582 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:43.184377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c681b6a3-cd96-4edc-b14d-8548a1f78582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:43.288350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:43.291048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 511/1025] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:43.574586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7177f13-49d9-490d-96ec-c4cfd7b77cf3 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:32:43.576119 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c7177f13-49d9-490d-96ec-c4cfd7b77cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:43.680976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7177f13-49d9-490d-96ec-c4cfd7b77cf3 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:32:43.731628 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:43.758353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccee251-18d6-4d6c-b213-2ee1f9e742f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:32:43.759052 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccee251-18d6-4d6c-b213-2ee1f9e742f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:43.759253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccee251-18d6-4d6c-b213-2ee1f9e742f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:43.759521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccee251-18d6-4d6c-b213-2ee1f9e742f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:43.761153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-bccee251-18d6-4d6c-b213-2ee1f9e742f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:43.874763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccee251-18d6-4d6c-b213-2ee1f9e742f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:32:43.876235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccee251-18d6-4d6c-b213-2ee1f9e742f2 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:32:43.895309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 515/1026] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:32:48.855315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78fa3c-1c6b-4f3e-b5d3-f79e0cf3130a None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:48.855834 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78fa3c-1c6b-4f3e-b5d3-f79e0cf3130a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:48.855834 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78fa3c-1c6b-4f3e-b5d3-f79e0cf3130a None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:48.857524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 512/1027] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:48.862869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabb44d-0585-4b4f-a546-e3d63c696393 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:48.863279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabb44d-0585-4b4f-a546-e3d63c696393 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:48.863479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabb44d-0585-4b4f-a546-e3d63c696393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:48.979638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ddabb44d-0585-4b4f-a546-e3d63c696393 None None] MFA Rules not processed for user `99ec4a4276f345c2998723e554ab9608`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:48.981025 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ddabb44d-0585-4b4f-a546-e3d63c696393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:49.038476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:49.180132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 516/1028] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:32:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:53.527891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be550500-d77b-465f-8e7b-d8f0ee1f9f38 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:53.528443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be550500-d77b-465f-8e7b-d8f0ee1f9f38 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:53.528443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be550500-d77b-465f-8e7b-d8f0ee1f9f38 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:53.530248 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 513/1029] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:32:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:53.535300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad098135-c554-4f5c-a527-60654bf203fb None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:53.535762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad098135-c554-4f5c-a527-60654bf203fb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:53.535857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad098135-c554-4f5c-a527-60654bf203fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:53.625011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ad098135-c554-4f5c-a527-60654bf203fb None None] MFA Rules not processed for user `99ec4a4276f345c2998723e554ab9608`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:53.626239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ad098135-c554-4f5c-a527-60654bf203fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:53.711380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:53.761832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 517/1030] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:32:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:54.962618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f036f9-8372-4299-bf98-86bf2fd9e634 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:54.963175 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f036f9-8372-4299-bf98-86bf2fd9e634 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:54.963175 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f036f9-8372-4299-bf98-86bf2fd9e634 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:54.964442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 514/1031] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:32:54.969446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb10be-c44c-4253-8f45-b37331962d5b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:54.969886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb10be-c44c-4253-8f45-b37331962d5b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:54.970068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb10be-c44c-4253-8f45-b37331962d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:55.048054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9adb10be-c44c-4253-8f45-b37331962d5b None None] MFA Rules not processed for user `99ec4a4276f345c2998723e554ab9608`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:55.049200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9adb10be-c44c-4253-8f45-b37331962d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:55.162908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 518/1032] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:32:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:32:59.714518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001f730-1331-47de-8cc9-9ffb62297267 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:59.716814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001f730-1331-47de-8cc9-9ffb62297267 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:59.716814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001f730-1331-47de-8cc9-9ffb62297267 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:59.717113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 515/1033] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:32:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:32:59.722893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c9128-23e8-4485-9ff3-832ef1c7565e None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:32:59.723750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c9128-23e8-4485-9ff3-832ef1c7565e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:32:59.724168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c9128-23e8-4485-9ff3-832ef1c7565e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:32:59.768985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:32:59.807878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-de4c9128-23e8-4485-9ff3-832ef1c7565e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:32:59.809345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-de4c9128-23e8-4485-9ff3-832ef1c7565e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:32:59.918877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 519/1034] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:00.227258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff435bb8-2e78-4370-9547-30f8a3201c89 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:00.227744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff435bb8-2e78-4370-9547-30f8a3201c89 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:00.229634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff435bb8-2e78-4370-9547-30f8a3201c89 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:00.233293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 516/1035] 10.222.0.17 () {54 vars in 961 bytes} [Tue Oct 18 11:33:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:00.241316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60397235-1f6f-4ef0-8334-225339dccbda None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:00.241828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60397235-1f6f-4ef0-8334-225339dccbda None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:00.242247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60397235-1f6f-4ef0-8334-225339dccbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:00.333186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-60397235-1f6f-4ef0-8334-225339dccbda None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:00.334405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-60397235-1f6f-4ef0-8334-225339dccbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:00.395759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:00.480821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 520/1036] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Oct 18 11:33:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:00.491076 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff435bb8-2e78-4370-9547-30f8a3201c89 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:00.493848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ff435bb8-2e78-4370-9547-30f8a3201c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:00.770809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff435bb8-2e78-4370-9547-30f8a3201c89 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:00.866218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:00.915898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183c9e0-38e4-4ed3-abbc-04de5c5d474c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:00.916712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183c9e0-38e4-4ed3-abbc-04de5c5d474c service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:00.917119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183c9e0-38e4-4ed3-abbc-04de5c5d474c service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:00.917558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183c9e0-38e4-4ed3-abbc-04de5c5d474c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:00.919295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5183c9e0-38e4-4ed3-abbc-04de5c5d474c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:01.043036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183c9e0-38e4-4ed3-abbc-04de5c5d474c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:01.044553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183c9e0-38e4-4ed3-abbc-04de5c5d474c service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:01.067150 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 517/1037] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:03.558790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23207-cd00-4b84-891c-b04e3550a746 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:03.559592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23207-cd00-4b84-891c-b04e3550a746 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:03.559592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23207-cd00-4b84-891c-b04e3550a746 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:03.562501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 521/1038] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:33:03.566889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a33c-92bb-482d-9d08-b4ee76935889 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:03.567373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a33c-92bb-482d-9d08-b4ee76935889 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:03.567373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a33c-92bb-482d-9d08-b4ee76935889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:03.648462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-a962a33c-92bb-482d-9d08-b4ee76935889 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:03.650181 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a962a33c-92bb-482d-9d08-b4ee76935889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:03.717762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:03.804590 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 518/1039] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:04.138573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc23207-cd00-4b84-891c-b04e3550a746 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:04.141037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-bdc23207-cd00-4b84-891c-b04e3550a746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:04.343306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc23207-cd00-4b84-891c-b04e3550a746 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:04.358425 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:04.503574 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf24fb6-db3d-40b7-9bb6-5dba2be735ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:04.504268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf24fb6-db3d-40b7-9bb6-5dba2be735ab service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:04.504614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf24fb6-db3d-40b7-9bb6-5dba2be735ab service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:04.504980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf24fb6-db3d-40b7-9bb6-5dba2be735ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:04.506743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-1bf24fb6-db3d-40b7-9bb6-5dba2be735ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:04.616796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf24fb6-db3d-40b7-9bb6-5dba2be735ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:04.620093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf24fb6-db3d-40b7-9bb6-5dba2be735ab service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:04.646561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 522/1040] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:07.286776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7c20e-089f-43c7-bf9c-4974daeb82ea None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:07.287437 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7c20e-089f-43c7-bf9c-4974daeb82ea None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:07.287538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7c20e-089f-43c7-bf9c-4974daeb82ea None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:07.289335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 519/1041] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:07.295039 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e970cd-2002-430c-a952-df7c9dabed68 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:07.295555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e970cd-2002-430c-a952-df7c9dabed68 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:07.295636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e970cd-2002-430c-a952-df7c9dabed68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:07.312418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:07.370706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-f8e970cd-2002-430c-a952-df7c9dabed68 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:07.371944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f8e970cd-2002-430c-a952-df7c9dabed68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:07.483517 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 523/1042] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:07.772592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab0c4f-b0f2-4f12-82ae-486ae136e058 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:07.773023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab0c4f-b0f2-4f12-82ae-486ae136e058 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:07.773408 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab0c4f-b0f2-4f12-82ae-486ae136e058 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:07.775003 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 520/1043] 10.222.0.17 () {54 vars in 961 bytes} [Tue Oct 18 11:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:07.787011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cea407-a07a-484c-b555-1c5a3ab5b253 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:07.787431 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cea407-a07a-484c-b555-1c5a3ab5b253 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:07.787809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cea407-a07a-484c-b555-1c5a3ab5b253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:07.871865 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-08cea407-a07a-484c-b555-1c5a3ab5b253 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:07.873448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-08cea407-a07a-484c-b555-1c5a3ab5b253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:07.913710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:08.018337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 524/1044] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Oct 18 11:33:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:08.026692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab0c4f-b0f2-4f12-82ae-486ae136e058 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:08.028381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9cab0c4f-b0f2-4f12-82ae-486ae136e058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:08.226545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab0c4f-b0f2-4f12-82ae-486ae136e058 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:08.251659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:08.361421 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca991cf-9874-48d2-bccb-50f1ab966e62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:08.362047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca991cf-9874-48d2-bccb-50f1ab966e62 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:08.362320 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca991cf-9874-48d2-bccb-50f1ab966e62 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:08.362517 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca991cf-9874-48d2-bccb-50f1ab966e62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:08.364348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9ca991cf-9874-48d2-bccb-50f1ab966e62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:08.372022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13052457-c9d0-48ab-a54b-f5b9b95674d3 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:08.373372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13052457-c9d0-48ab-a54b-f5b9b95674d3 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:08.373654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13052457-c9d0-48ab-a54b-f5b9b95674d3 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:08.376878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 525/1045] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:08.380474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff4d7d-9b8c-4024-bdbd-0a3719854d83 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:08.380750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff4d7d-9b8c-4024-bdbd-0a3719854d83 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:08.381299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff4d7d-9b8c-4024-bdbd-0a3719854d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:08.454873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-86ff4d7d-9b8c-4024-bdbd-0a3719854d83 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:08.456814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-86ff4d7d-9b8c-4024-bdbd-0a3719854d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:08.474489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca991cf-9874-48d2-bccb-50f1ab966e62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:08.476122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca991cf-9874-48d2-bccb-50f1ab966e62 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:08.497227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 521/1046] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:08.578121 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:08.588695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 526/1047] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:33:08.897656 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d401c-c0a3-43fb-992e-d5b518b3608d None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:08.898027 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d401c-c0a3-43fb-992e-d5b518b3608d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:08.898410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d401c-c0a3-43fb-992e-d5b518b3608d None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:08.900422 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 522/1048] 10.222.0.17 () {54 vars in 963 bytes} [Tue Oct 18 11:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:08.905357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a136a-13dd-49f5-9557-d7d5753dacd4 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:08.905647 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a136a-13dd-49f5-9557-d7d5753dacd4 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:08.905922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a136a-13dd-49f5-9557-d7d5753dacd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:08.997212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-be0a136a-13dd-49f5-9557-d7d5753dacd4 None None] MFA Rules not processed for user `bb051b00a16949798158bac1ac57d36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:08.998431 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-be0a136a-13dd-49f5-9557-d7d5753dacd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:09.092458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 527/1049] 10.222.0.17 () {58 vars in 1060 bytes} [Tue Oct 18 11:33:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:09.100222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d401c-c0a3-43fb-992e-d5b518b3608d None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:09.101852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2b9d401c-c0a3-43fb-992e-d5b518b3608d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:09.133810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:09.259367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d401c-c0a3-43fb-992e-d5b518b3608d None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:09.363968 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3512de1-711e-4994-a497-503d7a1f759a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:09.364585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3512de1-711e-4994-a497-503d7a1f759a service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:09.364874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3512de1-711e-4994-a497-503d7a1f759a service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:09.365273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3512de1-711e-4994-a497-503d7a1f759a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:09.366865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a3512de1-711e-4994-a497-503d7a1f759a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:09.509991 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:09.563499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3512de1-711e-4994-a497-503d7a1f759a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:09.565011 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3512de1-711e-4994-a497-503d7a1f759a service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:09.593321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 523/1050] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:10.860450 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935802bc-2b72-46a7-8033-56cec7aad080 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:10.861055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935802bc-2b72-46a7-8033-56cec7aad080 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:10.861055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935802bc-2b72-46a7-8033-56cec7aad080 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:10.862409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 528/1051] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:10.868375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a818da9-0ba6-4475-a95b-0424573af96c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:10.868932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a818da9-0ba6-4475-a95b-0424573af96c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:10.869160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a818da9-0ba6-4475-a95b-0424573af96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:10.946724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1a818da9-0ba6-4475-a95b-0424573af96c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:10.947957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1a818da9-0ba6-4475-a95b-0424573af96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:11.062387 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 524/1052] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:11.375485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935802bc-2b72-46a7-8033-56cec7aad080 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:11.377509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-935802bc-2b72-46a7-8033-56cec7aad080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:11.496394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:11.571475 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935802bc-2b72-46a7-8033-56cec7aad080 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:11.713635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d597352-ead9-418f-b27c-ddcbe90465b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:11.714348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d597352-ead9-418f-b27c-ddcbe90465b6 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:11.714525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d597352-ead9-418f-b27c-ddcbe90465b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:11.715166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d597352-ead9-418f-b27c-ddcbe90465b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:11.716657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9d597352-ead9-418f-b27c-ddcbe90465b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:11.781575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:11.833645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d597352-ead9-418f-b27c-ddcbe90465b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:11.835331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d597352-ead9-418f-b27c-ddcbe90465b6 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:11.857801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 529/1053] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:14.109437 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55861d-4b5d-4f02-9ff4-83e295aff7c3 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:14.110022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55861d-4b5d-4f02-9ff4-83e295aff7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:14.110022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55861d-4b5d-4f02-9ff4-83e295aff7c3 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:14.112883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 525/1054] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:14.116990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d80d8-5bb9-4dbe-b75f-9aeefe77f8aa None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:14.117408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d80d8-5bb9-4dbe-b75f-9aeefe77f8aa None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:14.117596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d80d8-5bb9-4dbe-b75f-9aeefe77f8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:14.196334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-5d3d80d8-5bb9-4dbe-b75f-9aeefe77f8aa None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:14.197670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5d3d80d8-5bb9-4dbe-b75f-9aeefe77f8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:14.308880 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 530/1055] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:14.595511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55861d-4b5d-4f02-9ff4-83e295aff7c3 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:14.597174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7b55861d-4b5d-4f02-9ff4-83e295aff7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:14.641490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:14.777562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55861d-4b5d-4f02-9ff4-83e295aff7c3 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:14.913028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:14.919558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d9d15-2805-41c8-93ef-36c92e5f7123 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:14.920572 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d9d15-2805-41c8-93ef-36c92e5f7123 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:14.920976 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d9d15-2805-41c8-93ef-36c92e5f7123 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:14.922058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d9d15-2805-41c8-93ef-36c92e5f7123 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:14.923904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-222d9d15-2805-41c8-93ef-36c92e5f7123 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:15.023712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d9d15-2805-41c8-93ef-36c92e5f7123 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:15.025582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d9d15-2805-41c8-93ef-36c92e5f7123 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:15.046411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 526/1056] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:17.307155 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ea740-0d68-4bd6-9192-b04605969b4e None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:17.307667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ea740-0d68-4bd6-9192-b04605969b4e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:17.307667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ea740-0d68-4bd6-9192-b04605969b4e None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:17.309282 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 531/1057] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:17.314460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae211c5-8b90-49b9-a4de-8f1d3137c28e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:17.314460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae211c5-8b90-49b9-a4de-8f1d3137c28e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:17.315084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae211c5-8b90-49b9-a4de-8f1d3137c28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:17.392480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-cae211c5-8b90-49b9-a4de-8f1d3137c28e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:17.393763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cae211c5-8b90-49b9-a4de-8f1d3137c28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:17.462619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:17.500770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 527/1058] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:17.878180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ea740-0d68-4bd6-9192-b04605969b4e None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:17.879726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6e4ea740-0d68-4bd6-9192-b04605969b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:17.911220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:18.039130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ea740-0d68-4bd6-9192-b04605969b4e None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:18.153650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:18.175483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a7f1d-2acf-41ae-9ed1-903f10b332ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:18.176112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a7f1d-2acf-41ae-9ed1-903f10b332ba service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:18.176389 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a7f1d-2acf-41ae-9ed1-903f10b332ba service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:18.176644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a7f1d-2acf-41ae-9ed1-903f10b332ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:18.178360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-271a7f1d-2acf-41ae-9ed1-903f10b332ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:18.277505 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a7f1d-2acf-41ae-9ed1-903f10b332ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:18.280100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a7f1d-2acf-41ae-9ed1-903f10b332ba service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:18.304972 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 532/1059] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:20.534818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d21c-1d7c-460f-a38e-c1712dc3f4d3 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:20.535400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d21c-1d7c-460f-a38e-c1712dc3f4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:20.535400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d21c-1d7c-460f-a38e-c1712dc3f4d3 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:20.537344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 528/1060] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:20.543789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb835889-c089-425b-aabb-b77a2f73349d None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:20.544244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb835889-c089-425b-aabb-b77a2f73349d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:20.544350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb835889-c089-425b-aabb-b77a2f73349d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:20.614789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-eb835889-c089-425b-aabb-b77a2f73349d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:20.616904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-eb835889-c089-425b-aabb-b77a2f73349d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:20.679909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:20.723185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 533/1061] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:21.017705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40d21c-1d7c-460f-a38e-c1712dc3f4d3 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:21.019325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ff40d21c-1d7c-460f-a38e-c1712dc3f4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:21.191693 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40d21c-1d7c-460f-a38e-c1712dc3f4d3 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:21.255993 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:21.331354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acff5d7-50be-4d71-9248-6b3a36b6c761 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:21.331983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acff5d7-50be-4d71-9248-6b3a36b6c761 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:21.332334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acff5d7-50be-4d71-9248-6b3a36b6c761 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:21.332534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acff5d7-50be-4d71-9248-6b3a36b6c761 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:21.334368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1acff5d7-50be-4d71-9248-6b3a36b6c761 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:21.438497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acff5d7-50be-4d71-9248-6b3a36b6c761 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:21.440267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acff5d7-50be-4d71-9248-6b3a36b6c761 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:21.462158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 529/1062] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:23.712180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02253a0f-f267-423e-8431-2105eb03db96 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:23.712794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02253a0f-f267-423e-8431-2105eb03db96 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:23.712794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02253a0f-f267-423e-8431-2105eb03db96 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:23.714095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 534/1063] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:23.719919 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f645f92-cd12-408e-89d9-8ada8aa58ae3 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:23.720480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f645f92-cd12-408e-89d9-8ada8aa58ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:23.720480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f645f92-cd12-408e-89d9-8ada8aa58ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:23.796826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-0f645f92-cd12-408e-89d9-8ada8aa58ae3 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:23.798132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0f645f92-cd12-408e-89d9-8ada8aa58ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:23.808241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:23.901131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 530/1064] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:24.195116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02253a0f-f267-423e-8431-2105eb03db96 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:24.196639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-02253a0f-f267-423e-8431-2105eb03db96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:24.345828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02253a0f-f267-423e-8431-2105eb03db96 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:24.395868 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:24.475608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd09582-dd72-46b4-847c-f3e6791b1ad8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:24.476142 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd09582-dd72-46b4-847c-f3e6791b1ad8 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:24.476421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd09582-dd72-46b4-847c-f3e6791b1ad8 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:24.476691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd09582-dd72-46b4-847c-f3e6791b1ad8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:24.478694 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ecd09582-dd72-46b4-847c-f3e6791b1ad8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:24.582054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd09582-dd72-46b4-847c-f3e6791b1ad8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:24.583680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd09582-dd72-46b4-847c-f3e6791b1ad8 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:24.606164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 535/1065] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:25.083252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735061-b42a-48df-8be7-39d5cccef7bd None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:25.083820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735061-b42a-48df-8be7-39d5cccef7bd None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:25.083820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735061-b42a-48df-8be7-39d5cccef7bd None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:25.085174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 531/1066] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:33:25.091197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef5219-c09b-41b2-a04c-d541447b46fd None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:25.091662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef5219-c09b-41b2-a04c-d541447b46fd None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:25.091754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef5219-c09b-41b2-a04c-d541447b46fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:25.160867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:25.164106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-f6ef5219-c09b-41b2-a04c-d541447b46fd None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:25.165364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f6ef5219-c09b-41b2-a04c-d541447b46fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:25.266928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 536/1067] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:25.567057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947ef8-52d7-4a16-b334-d31b70b83579 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:25.567343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947ef8-52d7-4a16-b334-d31b70b83579 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:25.567649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61947ef8-52d7-4a16-b334-d31b70b83579 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:25.569000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 532/1068] 10.222.0.17 () {54 vars in 962 bytes} [Tue Oct 18 11:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:25.575060 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba02818-9df3-4714-9eca-cde48c466393 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:25.575426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba02818-9df3-4714-9eca-cde48c466393 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:25.575769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba02818-9df3-4714-9eca-cde48c466393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:25.652387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-cba02818-9df3-4714-9eca-cde48c466393 None None] MFA Rules not processed for user `3d2a655f4d6f45039d641133e3b56592`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:25.653656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cba02818-9df3-4714-9eca-cde48c466393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:25.725783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:25.733055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 537/1069] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Oct 18 11:33:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:25.741035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947ef8-52d7-4a16-b334-d31b70b83579 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:25.742648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-61947ef8-52d7-4a16-b334-d31b70b83579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:25.847252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61947ef8-52d7-4a16-b334-d31b70b83579 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:25.885399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:25.923034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf144312-a98a-423a-af2a-484881e75b67 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d2a655f4d6f45039d641133e3b56592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:25.923645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf144312-a98a-423a-af2a-484881e75b67 service glance] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:25.923924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf144312-a98a-423a-af2a-484881e75b67 service glance] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:25.924197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf144312-a98a-423a-af2a-484881e75b67 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:25.925829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-bf144312-a98a-423a-af2a-484881e75b67 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:26.037699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf144312-a98a-423a-af2a-484881e75b67 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:26.039133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf144312-a98a-423a-af2a-484881e75b67 service glance] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:26.057663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 533/1070] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:26.810971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad921c-743c-4daa-9bf0-b89299cdb103 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:26.811467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad921c-743c-4daa-9bf0-b89299cdb103 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:26.811581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad921c-743c-4daa-9bf0-b89299cdb103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:26.830357 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d851b-1a4a-4b75-b0d2-82f600ad546b None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:26.830911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d851b-1a4a-4b75-b0d2-82f600ad546b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:26.830911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d851b-1a4a-4b75-b0d2-82f600ad546b None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:26.832803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 534/1071] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:33:26.838058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22078988-d67f-4ef4-95d9-e79c248ec2b2 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:26.838347 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22078988-d67f-4ef4-95d9-e79c248ec2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:26.838626 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22078988-d67f-4ef4-95d9-e79c248ec2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:26.886620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d4ad921c-743c-4daa-9bf0-b89299cdb103 None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:26.888417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d4ad921c-743c-4daa-9bf0-b89299cdb103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:26.908362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-22078988-d67f-4ef4-95d9-e79c248ec2b2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:26.909618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-22078988-d67f-4ef4-95d9-e79c248ec2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:26.928818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:26.979525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 538/1072] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:33:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:27.020109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 535/1073] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:33:27.025600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7df8d-0ee7-43b3-8f98-609d71badbfc None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:27.025956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7df8d-0ee7-43b3-8f98-609d71badbfc None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:27.026300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7df8d-0ee7-43b3-8f98-609d71badbfc None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:27.027719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 539/1074] 10.222.0.17 () {54 vars in 943 bytes} [Tue Oct 18 11:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:33:27.035569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222f607-7f11-4e27-9d0b-35d5b8255502 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:27.036114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222f607-7f11-4e27-9d0b-35d5b8255502 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:27.036623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222f607-7f11-4e27-9d0b-35d5b8255502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:27.126504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1222f607-7f11-4e27-9d0b-35d5b8255502 None None] MFA Rules not processed for user `4be9de8292214872a59c1466f7513539`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:27.127833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1222f607-7f11-4e27-9d0b-35d5b8255502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:27.216515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:27.243305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 536/1075] 10.222.0.17 () {58 vars in 1040 bytes} [Tue Oct 18 11:33:27 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:27.253344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7df8d-0ee7-43b3-8f98-609d71badbfc None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:27.254863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b1c7df8d-0ee7-43b3-8f98-609d71badbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:27.311168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222f607-7f11-4e27-9d0b-35d5b8255502 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:27.312391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1222f607-7f11-4e27-9d0b-35d5b8255502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:27.341875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:27.398957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7df8d-0ee7-43b3-8f98-609d71badbfc None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:27.416910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222f607-7f11-4e27-9d0b-35d5b8255502 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:27.504928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0b753-4f0b-45fc-8317-d542983902b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:27.505475 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0b753-4f0b-45fc-8317-d542983902b4 service swift] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:27.505750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0b753-4f0b-45fc-8317-d542983902b4 service swift] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:27.506015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0b753-4f0b-45fc-8317-d542983902b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:27.507658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8ad0b753-4f0b-45fc-8317-d542983902b4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:27.525589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:27.537366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc060263-4a11-439d-a60d-d207463e7653 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:27.537970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc060263-4a11-439d-a60d-d207463e7653 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:27.538244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc060263-4a11-439d-a60d-d207463e7653 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:27.538511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc060263-4a11-439d-a60d-d207463e7653 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:27.540134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cc060263-4a11-439d-a60d-d207463e7653 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:27.579056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:27.595685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0b753-4f0b-45fc-8317-d542983902b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:27.597358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0b753-4f0b-45fc-8317-d542983902b4 service swift] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:27.606768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 540/1076] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:33:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:33:27.633317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc060263-4a11-439d-a60d-d207463e7653 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:27.635096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc060263-4a11-439d-a60d-d207463e7653 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:27.655293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 537/1077] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:33:29.900410 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb2816-8732-4740-8660-7a8b64f0ecaf None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:29.901045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb2816-8732-4740-8660-7a8b64f0ecaf None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:29.901045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb2816-8732-4740-8660-7a8b64f0ecaf None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:29.902372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 541/1078] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:29.907879 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4e664-1ab5-41c2-9d96-56911021bfa0 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:29.908340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4e664-1ab5-41c2-9d96-56911021bfa0 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:29.908436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4e664-1ab5-41c2-9d96-56911021bfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:29.985314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e2c4e664-1ab5-41c2-9d96-56911021bfa0 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:29.985314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e2c4e664-1ab5-41c2-9d96-56911021bfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:30.057427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:30.103899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 538/1079] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:30.399809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb2816-8732-4740-8660-7a8b64f0ecaf None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:30.403069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-edbb2816-8732-4740-8660-7a8b64f0ecaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:30.582574 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb2816-8732-4740-8660-7a8b64f0ecaf None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:30.656176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:30.718526 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978aa4c-a8ec-4eae-ac47-97efbcb794ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:30.719172 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978aa4c-a8ec-4eae-ac47-97efbcb794ab service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:30.719446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978aa4c-a8ec-4eae-ac47-97efbcb794ab service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:30.719711 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978aa4c-a8ec-4eae-ac47-97efbcb794ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:30.721438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3978aa4c-a8ec-4eae-ac47-97efbcb794ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:30.825464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978aa4c-a8ec-4eae-ac47-97efbcb794ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:30.827591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978aa4c-a8ec-4eae-ac47-97efbcb794ab service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:30.871861 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 542/1080] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:33.116024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8374c27-d009-4402-9e0d-53ca30634418 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:33.116709 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8374c27-d009-4402-9e0d-53ca30634418 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:33.117067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8374c27-d009-4402-9e0d-53ca30634418 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:33.120064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 539/1081] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:33.129588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1ffa3-4159-49bf-9b27-f67a5401eb27 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:33.129588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1ffa3-4159-49bf-9b27-f67a5401eb27 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:33.129588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1ffa3-4159-49bf-9b27-f67a5401eb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:33.209329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-22a1ffa3-4159-49bf-9b27-f67a5401eb27 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:33.210637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-22a1ffa3-4159-49bf-9b27-f67a5401eb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:33.233171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:33.318199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 543/1082] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:33.609713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8374c27-d009-4402-9e0d-53ca30634418 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:33.611577 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a8374c27-d009-4402-9e0d-53ca30634418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:33.781026 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8374c27-d009-4402-9e0d-53ca30634418 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:33.844133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:33.923688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b53c75-0e88-41ba-9a64-c58968b9e9fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:33.924354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b53c75-0e88-41ba-9a64-c58968b9e9fd service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:33.924636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b53c75-0e88-41ba-9a64-c58968b9e9fd service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:33.924945 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b53c75-0e88-41ba-9a64-c58968b9e9fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:33.926701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-64b53c75-0e88-41ba-9a64-c58968b9e9fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:34.038395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b53c75-0e88-41ba-9a64-c58968b9e9fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:34.040067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b53c75-0e88-41ba-9a64-c58968b9e9fd service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:34.063540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 540/1083] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:36.317092 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bc816-265d-489b-acb8-d02c909c0a6b None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:36.317692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bc816-265d-489b-acb8-d02c909c0a6b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:36.317692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bc816-265d-489b-acb8-d02c909c0a6b None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:36.319012 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 544/1084] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:36.325490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16d36a-4cd4-4406-a3b1-d81373985da2 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:36.326068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16d36a-4cd4-4406-a3b1-d81373985da2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:36.326068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16d36a-4cd4-4406-a3b1-d81373985da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:36.403304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-df16d36a-4cd4-4406-a3b1-d81373985da2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:36.404666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-df16d36a-4cd4-4406-a3b1-d81373985da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:36.407889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:36.512288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 541/1085] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:36.832545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bc816-265d-489b-acb8-d02c909c0a6b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:36.834501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-358bc816-265d-489b-acb8-d02c909c0a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:37.001068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bc816-265d-489b-acb8-d02c909c0a6b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:37.005691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:37.148740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92775931-29a3-494f-9478-ee25fc5572f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:37.149418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92775931-29a3-494f-9478-ee25fc5572f6 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:37.149693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92775931-29a3-494f-9478-ee25fc5572f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:37.149961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92775931-29a3-494f-9478-ee25fc5572f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:37.151687 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-92775931-29a3-494f-9478-ee25fc5572f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:37.271207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92775931-29a3-494f-9478-ee25fc5572f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:37.272825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92775931-29a3-494f-9478-ee25fc5572f6 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:37.298585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 545/1086] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:46.509803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=41 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:47.291166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:50.745619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f831dc0-f71a-470b-ab94-2b9f64fd56d2 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:50.746188 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f831dc0-f71a-470b-ab94-2b9f64fd56d2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:50.746188 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f831dc0-f71a-470b-ab94-2b9f64fd56d2 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:50.747504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 542/1087] 10.222.0.17 () {54 vars in 918 bytes} [Tue Oct 18 11:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:50.752403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc740f50-6dfb-45d7-b28e-7d79c463b6e0 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:50.752892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc740f50-6dfb-45d7-b28e-7d79c463b6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:50.753014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc740f50-6dfb-45d7-b28e-7d79c463b6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:50.828058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-bc740f50-6dfb-45d7-b28e-7d79c463b6e0 None None] MFA Rules not processed for user `99ec4a4276f345c2998723e554ab9608`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:50.829241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-bc740f50-6dfb-45d7-b28e-7d79c463b6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:50.935091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 546/1088] 10.222.0.17 () {58 vars in 1015 bytes} [Tue Oct 18 11:33:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:50.949854 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06958a8-3141-4ab7-8687-9c0a8f85d7fc None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:50.950135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06958a8-3141-4ab7-8687-9c0a8f85d7fc None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:50.950404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06958a8-3141-4ab7-8687-9c0a8f85d7fc None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:50.951701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 543/1089] 10.222.0.17 () {54 vars in 918 bytes} [Tue Oct 18 11:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:50.956425 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cfae9-c665-4c9c-951e-9d7950e96f2b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:50.956706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cfae9-c665-4c9c-951e-9d7950e96f2b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:50.957018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cfae9-c665-4c9c-951e-9d7950e96f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:51.029561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:51.032181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-0a7cfae9-c665-4c9c-951e-9d7950e96f2b None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:51.034036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0a7cfae9-c665-4c9c-951e-9d7950e96f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:51.148316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 547/1090] 10.222.0.17 () {58 vars in 1015 bytes} [Tue Oct 18 11:33:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:33:53.806637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f444937-e63b-4535-bb75-6efa3ebaee5b None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:53.807258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f444937-e63b-4535-bb75-6efa3ebaee5b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:53.807258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f444937-e63b-4535-bb75-6efa3ebaee5b None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:53.809119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 544/1091] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:33:53.815246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6690788-4dbc-428b-ace4-a1b7dd355fc3 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:53.816063 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6690788-4dbc-428b-ace4-a1b7dd355fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:53.816226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6690788-4dbc-428b-ace4-a1b7dd355fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:53.892474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d6690788-4dbc-428b-ace4-a1b7dd355fc3 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:53.893791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d6690788-4dbc-428b-ace4-a1b7dd355fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:53.955907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:54.002807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 548/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:33:54.296806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f444937-e63b-4535-bb75-6efa3ebaee5b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:54.298565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5f444937-e63b-4535-bb75-6efa3ebaee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:54.536579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f444937-e63b-4535-bb75-6efa3ebaee5b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:54.639640 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:54.687056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ec02c-f89a-4c42-b4b7-6c60dc8b3c8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:54.687737 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ec02c-f89a-4c42-b4b7-6c60dc8b3c8c service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:54.688058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ec02c-f89a-4c42-b4b7-6c60dc8b3c8c service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:54.688373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ec02c-f89a-4c42-b4b7-6c60dc8b3c8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:54.690348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-050ec02c-f89a-4c42-b4b7-6c60dc8b3c8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:54.796042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ec02c-f89a-4c42-b4b7-6c60dc8b3c8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:54.797570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ec02c-f89a-4c42-b4b7-6c60dc8b3c8c service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:54.816899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 545/1093] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:56.046987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815a90a-b476-4076-a879-34904029c865 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:56.047668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815a90a-b476-4076-a879-34904029c865 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:56.047668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815a90a-b476-4076-a879-34904029c865 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:56.049037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 549/1094] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:33:56.055896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd77179-b6ba-4ca0-ba4b-ca00e4c95b92 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:56.056431 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd77179-b6ba-4ca0-ba4b-ca00e4c95b92 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:56.056431 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd77179-b6ba-4ca0-ba4b-ca00e4c95b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:56.165425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6fd77179-b6ba-4ca0-ba4b-ca00e4c95b92 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:56.166767 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6fd77179-b6ba-4ca0-ba4b-ca00e4c95b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:56.205839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:56.296591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 546/1095] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:56.491044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815a90a-b476-4076-a879-34904029c865 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:56.492641 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e815a90a-b476-4076-a879-34904029c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:56.681623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815a90a-b476-4076-a879-34904029c865 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:56.730703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:56.826095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eac2ac-21d4-449f-85dc-4fb28df079e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:56.826724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eac2ac-21d4-449f-85dc-4fb28df079e7 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:56.827024 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eac2ac-21d4-449f-85dc-4fb28df079e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:56.827391 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eac2ac-21d4-449f-85dc-4fb28df079e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:56.829776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-48eac2ac-21d4-449f-85dc-4fb28df079e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:56.940199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eac2ac-21d4-449f-85dc-4fb28df079e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:56.942056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eac2ac-21d4-449f-85dc-4fb28df079e7 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:56.965286 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 550/1096] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:33:58.199786 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a864aed-2892-4b55-9813-5f18fefab147 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:58.200643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a864aed-2892-4b55-9813-5f18fefab147 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:58.200812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a864aed-2892-4b55-9813-5f18fefab147 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:58.203199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 547/1097] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:33:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:33:58.213459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8679b-19bf-4107-aaf6-b46e06aa728b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:58.214708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8679b-19bf-4107-aaf6-b46e06aa728b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:58.214708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8679b-19bf-4107-aaf6-b46e06aa728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:58.278934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:58.304701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-61d8679b-19bf-4107-aaf6-b46e06aa728b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:33:58.306062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-61d8679b-19bf-4107-aaf6-b46e06aa728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:58.419174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 551/1098] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:33:58.582052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a864aed-2892-4b55-9813-5f18fefab147 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:33:58.583742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9a864aed-2892-4b55-9813-5f18fefab147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:58.760716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a864aed-2892-4b55-9813-5f18fefab147 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:33:58.774560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:33:58.895335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae5fe3-6a04-4f49-9442-de7a46397b70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:33:58.896118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae5fe3-6a04-4f49-9442-de7a46397b70 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:33:58.896498 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae5fe3-6a04-4f49-9442-de7a46397b70 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:33:58.896923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae5fe3-6a04-4f49-9442-de7a46397b70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:33:58.899026 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e2ae5fe3-6a04-4f49-9442-de7a46397b70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:33:59.001537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae5fe3-6a04-4f49-9442-de7a46397b70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:33:59.003359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae5fe3-6a04-4f49-9442-de7a46397b70 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:33:59.025292 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 548/1099] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:00.258547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a48995-19eb-4cd3-98c6-d6ce25df8898 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:00.259110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a48995-19eb-4cd3-98c6-d6ce25df8898 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:00.259110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a48995-19eb-4cd3-98c6-d6ce25df8898 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:00.260580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 552/1100] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:00.267850 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5cb8d-ddc8-406b-94ba-663630b6c104 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:00.268687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5cb8d-ddc8-406b-94ba-663630b6c104 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:00.268890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5cb8d-ddc8-406b-94ba-663630b6c104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:00.289904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:00.351233 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-5fb5cb8d-ddc8-406b-94ba-663630b6c104 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:00.352720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5fb5cb8d-ddc8-406b-94ba-663630b6c104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:00.497189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 549/1101] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:34:00.664832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a48995-19eb-4cd3-98c6-d6ce25df8898 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:00.666641 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a2a48995-19eb-4cd3-98c6-d6ce25df8898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:00.847613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:00.890935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a48995-19eb-4cd3-98c6-d6ce25df8898 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:01.027516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228ce23-2524-4d2f-88b0-2bdc9b558ead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:01.028186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228ce23-2524-4d2f-88b0-2bdc9b558ead service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:01.028519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228ce23-2524-4d2f-88b0-2bdc9b558ead service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:01.028888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228ce23-2524-4d2f-88b0-2bdc9b558ead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:01.030763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c228ce23-2524-4d2f-88b0-2bdc9b558ead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:01.121955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:01.138154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228ce23-2524-4d2f-88b0-2bdc9b558ead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:01.139763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228ce23-2524-4d2f-88b0-2bdc9b558ead service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:01.164913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 553/1102] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:02.397940 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e8163-07d9-4672-afc5-f69d875c25db None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:02.398803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e8163-07d9-4672-afc5-f69d875c25db None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:02.398803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e8163-07d9-4672-afc5-f69d875c25db None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:02.400421 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 550/1103] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:34:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:02.411130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ec5d6-1cdb-4e68-86f9-bde36991348b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:02.411698 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ec5d6-1cdb-4e68-86f9-bde36991348b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:02.411976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ec5d6-1cdb-4e68-86f9-bde36991348b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:02.503762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-548ec5d6-1cdb-4e68-86f9-bde36991348b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:02.504924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-548ec5d6-1cdb-4e68-86f9-bde36991348b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:02.631015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 554/1104] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:02.790008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e8163-07d9-4672-afc5-f69d875c25db None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:02.791533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-430e8163-07d9-4672-afc5-f69d875c25db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:02.892038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:02.964179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e8163-07d9-4672-afc5-f69d875c25db None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:03.096799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355e209-4aa4-4410-bc69-41560ab9437f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:03.097429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355e209-4aa4-4410-bc69-41560ab9437f service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:03.097765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355e209-4aa4-4410-bc69-41560ab9437f service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:03.098088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355e209-4aa4-4410-bc69-41560ab9437f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:03.100046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8355e209-4aa4-4410-bc69-41560ab9437f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:03.157690 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:03.210309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355e209-4aa4-4410-bc69-41560ab9437f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:03.212063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355e209-4aa4-4410-bc69-41560ab9437f service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:03.234915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 551/1105] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:04.378875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d37e8-c348-471a-bee6-33bfb6e82248 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:04.379555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d37e8-c348-471a-bee6-33bfb6e82248 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:04.379555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d37e8-c348-471a-bee6-33bfb6e82248 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:04.380760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 555/1106] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:04.388048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba5504-15f8-4a63-9429-75b3f590b438 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:04.388917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba5504-15f8-4a63-9429-75b3f590b438 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:04.389081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba5504-15f8-4a63-9429-75b3f590b438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:04.475374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-33ba5504-15f8-4a63-9429-75b3f590b438 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:04.476784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-33ba5504-15f8-4a63-9429-75b3f590b438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:04.599948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 552/1107] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:04.764317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d37e8-c348-471a-bee6-33bfb6e82248 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:04.765856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4e2d37e8-c348-471a-bee6-33bfb6e82248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:04.819269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:04.927399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d37e8-c348-471a-bee6-33bfb6e82248 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:05.049543 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdcdc4-ae14-41ee-81c6-546e079a7d79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:05.050061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdcdc4-ae14-41ee-81c6-546e079a7d79 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:05.050353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdcdc4-ae14-41ee-81c6-546e079a7d79 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:05.050619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdcdc4-ae14-41ee-81c6-546e079a7d79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:05.052305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9bbdcdc4-ae14-41ee-81c6-546e079a7d79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:05.060470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:05.157955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdcdc4-ae14-41ee-81c6-546e079a7d79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:05.159646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdcdc4-ae14-41ee-81c6-546e079a7d79 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:05.183239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 556/1108] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:06.349223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83b410-36ee-4013-8cb9-de4d5299bf8b None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:06.350103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83b410-36ee-4013-8cb9-de4d5299bf8b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:06.350103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83b410-36ee-4013-8cb9-de4d5299bf8b None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:06.351105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 553/1109] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:06.358596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d02ce-c448-4b9a-94b6-507cc074d3c7 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:06.359137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d02ce-c448-4b9a-94b6-507cc074d3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:06.359237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d02ce-c448-4b9a-94b6-507cc074d3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:06.438267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-1a8d02ce-c448-4b9a-94b6-507cc074d3c7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:06.439445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-1a8d02ce-c448-4b9a-94b6-507cc074d3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:06.513467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:06.545285 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 557/1110] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:06.698270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83b410-36ee-4013-8cb9-de4d5299bf8b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:06.699987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2d83b410-36ee-4013-8cb9-de4d5299bf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:06.719923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:06.862832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83b410-36ee-4013-8cb9-de4d5299bf8b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:06.956760 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:06.985647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce411a-cf7d-458b-981b-186e7f8f1baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:06.986261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce411a-cf7d-458b-981b-186e7f8f1baa service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:06.986546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce411a-cf7d-458b-981b-186e7f8f1baa service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:06.986811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce411a-cf7d-458b-981b-186e7f8f1baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:06.988430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d2ce411a-cf7d-458b-981b-186e7f8f1baa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:07.084981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce411a-cf7d-458b-981b-186e7f8f1baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:07.086639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce411a-cf7d-458b-981b-186e7f8f1baa service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:07.106649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 554/1111] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:08.206778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c1f10-34c7-40f7-b3a4-1b0dec4e4266 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:08.207434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c1f10-34c7-40f7-b3a4-1b0dec4e4266 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:08.207434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c1f10-34c7-40f7-b3a4-1b0dec4e4266 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:08.208732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 558/1112] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:08.216675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f54257-471e-4c0c-bd4c-48a4eeabcca2 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:08.217665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f54257-471e-4c0c-bd4c-48a4eeabcca2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:08.217665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f54257-471e-4c0c-bd4c-48a4eeabcca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:08.315553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-86f54257-471e-4c0c-bd4c-48a4eeabcca2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:08.316784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-86f54257-471e-4c0c-bd4c-48a4eeabcca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:08.367728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:08.425696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 555/1113] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:08.586887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c1f10-34c7-40f7-b3a4-1b0dec4e4266 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:08.588676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-177c1f10-34c7-40f7-b3a4-1b0dec4e4266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:08.768081 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c1f10-34c7-40f7-b3a4-1b0dec4e4266 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:08.848253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:08.913002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8167f15-6891-42ee-a139-92ef2997972c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:08.913616 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8167f15-6891-42ee-a139-92ef2997972c service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:08.913895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8167f15-6891-42ee-a139-92ef2997972c service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:08.914166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8167f15-6891-42ee-a139-92ef2997972c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:08.916384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c8167f15-6891-42ee-a139-92ef2997972c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:09.048686 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8167f15-6891-42ee-a139-92ef2997972c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:09.050831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8167f15-6891-42ee-a139-92ef2997972c service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:09.074045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 559/1114] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:34:10.187643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c679ef-c865-43be-bd4d-817e3fb7a13a None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:10.188132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c679ef-c865-43be-bd4d-817e3fb7a13a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:10.188251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c679ef-c865-43be-bd4d-817e3fb7a13a None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:10.189595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 556/1115] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:10.196311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d861974-fa43-471e-b1d9-ed6b5653d6e8 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:10.196823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d861974-fa43-471e-b1d9-ed6b5653d6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:10.196974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d861974-fa43-471e-b1d9-ed6b5653d6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:10.300334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9d861974-fa43-471e-b1d9-ed6b5653d6e8 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:10.301663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9d861974-fa43-471e-b1d9-ed6b5653d6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:10.303837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:10.408418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 560/1116] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:10.576230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c679ef-c865-43be-bd4d-817e3fb7a13a None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:10.578049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a7c679ef-c865-43be-bd4d-817e3fb7a13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:10.735821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c679ef-c865-43be-bd4d-817e3fb7a13a None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:10.767706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:10.880930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030a276-a0da-48bf-a66c-5efd23c756dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:10.881657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030a276-a0da-48bf-a66c-5efd23c756dd service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:10.881840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030a276-a0da-48bf-a66c-5efd23c756dd service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:10.882082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030a276-a0da-48bf-a66c-5efd23c756dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:10.883697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2030a276-a0da-48bf-a66c-5efd23c756dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:10.981368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030a276-a0da-48bf-a66c-5efd23c756dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:10.982958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030a276-a0da-48bf-a66c-5efd23c756dd service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:11.001753 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 557/1117] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:13.188031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083c532-715e-4fd5-950d-14e99977d8eb None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:13.188658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083c532-715e-4fd5-950d-14e99977d8eb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:13.188658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083c532-715e-4fd5-950d-14e99977d8eb None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:13.189959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 561/1118] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:34:13.195870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7cbf1-c312-4723-87c4-fe21b6dcf10b None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:13.196421 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7cbf1-c312-4723-87c4-fe21b6dcf10b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:13.196421 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7cbf1-c312-4723-87c4-fe21b6dcf10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:13.246172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:13.286098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-99c7cbf1-c312-4723-87c4-fe21b6dcf10b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:13.287442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-99c7cbf1-c312-4723-87c4-fe21b6dcf10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:13.395942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 558/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:13.706763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083c532-715e-4fd5-950d-14e99977d8eb None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:13.708436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b083c532-715e-4fd5-950d-14e99977d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:13.827532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083c532-715e-4fd5-950d-14e99977d8eb None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:13.864165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:13.909930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca53ca9-dff2-4ae7-9129-64d20505d154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:13.910547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca53ca9-dff2-4ae7-9129-64d20505d154 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:13.910821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca53ca9-dff2-4ae7-9129-64d20505d154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:13.911088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca53ca9-dff2-4ae7-9129-64d20505d154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:13.912784 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-dca53ca9-dff2-4ae7-9129-64d20505d154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:14.033311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca53ca9-dff2-4ae7-9129-64d20505d154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:14.034664 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca53ca9-dff2-4ae7-9129-64d20505d154 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:14.054776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 562/1120] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:16.468596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffbb4b-8649-4017-bfcf-d3e191baef3a None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:16.469186 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffbb4b-8649-4017-bfcf-d3e191baef3a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:16.469332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffbb4b-8649-4017-bfcf-d3e191baef3a None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:16.471179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 559/1121] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:16.477209 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46176666-328e-4ec6-a430-cd8e9ec13123 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:16.477756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46176666-328e-4ec6-a430-cd8e9ec13123 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:16.477889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46176666-328e-4ec6-a430-cd8e9ec13123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:16.559078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-46176666-328e-4ec6-a430-cd8e9ec13123 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:16.560319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-46176666-328e-4ec6-a430-cd8e9ec13123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:16.575682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:16.671351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 563/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:16.971604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffbb4b-8649-4017-bfcf-d3e191baef3a None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:16.973259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-43ffbb4b-8649-4017-bfcf-d3e191baef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:17.098610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffbb4b-8649-4017-bfcf-d3e191baef3a None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:17.114365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:17.202627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73306eee-0978-462d-87ce-3a17015d5d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:17.203369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73306eee-0978-462d-87ce-3a17015d5d15 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:17.203681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73306eee-0978-462d-87ce-3a17015d5d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:17.204074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73306eee-0978-462d-87ce-3a17015d5d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:17.205768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-73306eee-0978-462d-87ce-3a17015d5d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:17.333628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73306eee-0978-462d-87ce-3a17015d5d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:17.335164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73306eee-0978-462d-87ce-3a17015d5d15 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:17.355573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 560/1123] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:20.048902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af85a0-a6b5-4ba7-aa72-e3a2e463c287 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:20.049838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af85a0-a6b5-4ba7-aa72-e3a2e463c287 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:20.049838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af85a0-a6b5-4ba7-aa72-e3a2e463c287 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:20.050971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 564/1124] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:20.057004 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121460-6989-4319-989a-21549a6821a5 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:20.057583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121460-6989-4319-989a-21549a6821a5 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:20.057583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121460-6989-4319-989a-21549a6821a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:20.112990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:20.133730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-8d121460-6989-4319-989a-21549a6821a5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:20.134999 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8d121460-6989-4319-989a-21549a6821a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:20.249656 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 561/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:34:20.541764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af85a0-a6b5-4ba7-aa72-e3a2e463c287 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:20.543632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c7af85a0-a6b5-4ba7-aa72-e3a2e463c287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:20.663832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af85a0-a6b5-4ba7-aa72-e3a2e463c287 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:20.708809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:20.743069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc79bc1-c4da-4ab7-a5cf-1c6114df7189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:20.743660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc79bc1-c4da-4ab7-a5cf-1c6114df7189 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:20.743932 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc79bc1-c4da-4ab7-a5cf-1c6114df7189 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:20.744199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc79bc1-c4da-4ab7-a5cf-1c6114df7189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:20.746343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2fc79bc1-c4da-4ab7-a5cf-1c6114df7189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:20.860579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc79bc1-c4da-4ab7-a5cf-1c6114df7189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:20.861977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc79bc1-c4da-4ab7-a5cf-1c6114df7189 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:20.880383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 565/1126] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:34:23.176046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc94f-8638-42c8-b23b-19a6cc12756b None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:23.176602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc94f-8638-42c8-b23b-19a6cc12756b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:23.176602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc94f-8638-42c8-b23b-19a6cc12756b None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:23.178008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 562/1127] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:23.184614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2f062-b532-4f43-aa49-7a82d6dfba91 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:23.185096 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2f062-b532-4f43-aa49-7a82d6dfba91 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:23.185193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2f062-b532-4f43-aa49-7a82d6dfba91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:23.252864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-0fc2f062-b532-4f43-aa49-7a82d6dfba91 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:23.254086 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0fc2f062-b532-4f43-aa49-7a82d6dfba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:23.274753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:23.353185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 566/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:23.640496 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cc94f-8638-42c8-b23b-19a6cc12756b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:23.642049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5d5cc94f-8638-42c8-b23b-19a6cc12756b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:23.746262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cc94f-8638-42c8-b23b-19a6cc12756b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:23.766561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:23.821822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3af56a-1e50-4ec3-b86a-6bbc3522d128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:23.822466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3af56a-1e50-4ec3-b86a-6bbc3522d128 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:23.823325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3af56a-1e50-4ec3-b86a-6bbc3522d128 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:23.823670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3af56a-1e50-4ec3-b86a-6bbc3522d128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:23.825567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-aa3af56a-1e50-4ec3-b86a-6bbc3522d128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:23.938926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3af56a-1e50-4ec3-b86a-6bbc3522d128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:23.940287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3af56a-1e50-4ec3-b86a-6bbc3522d128 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:23.958686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 563/1129] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:26.181102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca9166-27fa-4984-b612-7622c1954e54 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:26.181102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca9166-27fa-4984-b612-7622c1954e54 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:26.181102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca9166-27fa-4984-b612-7622c1954e54 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:26.182368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 567/1130] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:26.188347 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa5c1d-c33c-4ade-b20f-2271205295b2 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:26.188933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa5c1d-c33c-4ade-b20f-2271205295b2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:26.188933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa5c1d-c33c-4ade-b20f-2271205295b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:26.259257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:26.263070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c7fa5c1d-c33c-4ade-b20f-2271205295b2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:26.265468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c7fa5c1d-c33c-4ade-b20f-2271205295b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:26.364092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 564/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:26.646836 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca9166-27fa-4984-b612-7622c1954e54 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:26.648580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-37ca9166-27fa-4984-b612-7622c1954e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:26.751389 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca9166-27fa-4984-b612-7622c1954e54 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:26.801198 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:26.829209 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ede390-971b-47ed-a148-27f58ca207ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:26.829928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ede390-971b-47ed-a148-27f58ca207ae service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:26.830237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ede390-971b-47ed-a148-27f58ca207ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:26.830725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ede390-971b-47ed-a148-27f58ca207ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:26.832901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-71ede390-971b-47ed-a148-27f58ca207ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:26.943456 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ede390-971b-47ed-a148-27f58ca207ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:26.944863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ede390-971b-47ed-a148-27f58ca207ae service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:26.962529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 568/1132] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:29.218703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926dd51-25cb-40d2-90fc-b4deade255aa None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:29.219272 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926dd51-25cb-40d2-90fc-b4deade255aa None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:29.219272 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926dd51-25cb-40d2-90fc-b4deade255aa None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:29.220568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 565/1133] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:29.226182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda08d36-4241-40f1-9d61-a71d84bcb688 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:29.226892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda08d36-4241-40f1-9d61-a71d84bcb688 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:29.227057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda08d36-4241-40f1-9d61-a71d84bcb688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:29.303011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-eda08d36-4241-40f1-9d61-a71d84bcb688 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:29.304465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-eda08d36-4241-40f1-9d61-a71d84bcb688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:29.339161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:29.414696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 569/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:29.705072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926dd51-25cb-40d2-90fc-b4deade255aa None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:29.706653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e926dd51-25cb-40d2-90fc-b4deade255aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:29.810697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926dd51-25cb-40d2-90fc-b4deade255aa None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:29.839720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:29.887352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc12418-107a-4a71-a353-f8f4a56c6e47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:29.887956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc12418-107a-4a71-a353-f8f4a56c6e47 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:29.888229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc12418-107a-4a71-a353-f8f4a56c6e47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:29.888499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc12418-107a-4a71-a353-f8f4a56c6e47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:29.890112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-bfc12418-107a-4a71-a353-f8f4a56c6e47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:29.997814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc12418-107a-4a71-a353-f8f4a56c6e47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:29.999145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc12418-107a-4a71-a353-f8f4a56c6e47 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:30.016832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 566/1135] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:32.252952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413c3c9-a002-47d5-ba05-3c93588aac5b None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:32.253523 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413c3c9-a002-47d5-ba05-3c93588aac5b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:32.253523 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413c3c9-a002-47d5-ba05-3c93588aac5b None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:32.254878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 570/1136] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:32.261487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411865ed-1724-4b03-9adb-e408e5afac46 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:32.262023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411865ed-1724-4b03-9adb-e408e5afac46 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:32.262023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411865ed-1724-4b03-9adb-e408e5afac46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:32.338605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-411865ed-1724-4b03-9adb-e408e5afac46 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:32.339862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-411865ed-1724-4b03-9adb-e408e5afac46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:32.349466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:32.444288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 567/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:32.727856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413c3c9-a002-47d5-ba05-3c93588aac5b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:32.729466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f413c3c9-a002-47d5-ba05-3c93588aac5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:32.848377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413c3c9-a002-47d5-ba05-3c93588aac5b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:32.906990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:32.925925 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacea922-92d4-4749-b96a-28f2ff3ea240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:32.926552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacea922-92d4-4749-b96a-28f2ff3ea240 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:32.926804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacea922-92d4-4749-b96a-28f2ff3ea240 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:32.927071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacea922-92d4-4749-b96a-28f2ff3ea240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:32.928673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-bacea922-92d4-4749-b96a-28f2ff3ea240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:33.041239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacea922-92d4-4749-b96a-28f2ff3ea240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:33.042605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacea922-92d4-4749-b96a-28f2ff3ea240 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:33.061804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 571/1138] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:35.320501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7ab93-f200-4ec1-8a29-4d7e06920002 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:35.321161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7ab93-f200-4ec1-8a29-4d7e06920002 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:35.321161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7ab93-f200-4ec1-8a29-4d7e06920002 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:35.323013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 568/1139] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:35.329199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76429762-fe19-45fb-86ef-850f77a698d9 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:35.329709 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76429762-fe19-45fb-86ef-850f77a698d9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:35.329812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76429762-fe19-45fb-86ef-850f77a698d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:35.400064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-76429762-fe19-45fb-86ef-850f77a698d9 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:35.401943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-76429762-fe19-45fb-86ef-850f77a698d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:35.442074 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:35.502155 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 572/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:35.788018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7ab93-f200-4ec1-8a29-4d7e06920002 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:35.789582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b0f7ab93-f200-4ec1-8a29-4d7e06920002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:35.917869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7ab93-f200-4ec1-8a29-4d7e06920002 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:35.965124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:36.004283 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a8bf4-e1a3-4ca5-bc11-4fdc93a39b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:36.005075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a8bf4-e1a3-4ca5-bc11-4fdc93a39b23 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:36.005433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a8bf4-e1a3-4ca5-bc11-4fdc93a39b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:36.005809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a8bf4-e1a3-4ca5-bc11-4fdc93a39b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:36.007553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-364a8bf4-e1a3-4ca5-bc11-4fdc93a39b23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:36.121248 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a8bf4-e1a3-4ca5-bc11-4fdc93a39b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:36.122724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a8bf4-e1a3-4ca5-bc11-4fdc93a39b23 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:36.143122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 569/1141] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:38.495291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f286ba-0319-45d0-9da0-380012a847b8 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:38.496036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f286ba-0319-45d0-9da0-380012a847b8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:38.496036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f286ba-0319-45d0-9da0-380012a847b8 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:38.497322 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 573/1142] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:38.506488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f92a-0593-4e8c-a124-6d807cb58a2c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:38.507024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f92a-0593-4e8c-a124-6d807cb58a2c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:38.507141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f92a-0593-4e8c-a124-6d807cb58a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:38.592856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-9c04f92a-0593-4e8c-a124-6d807cb58a2c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:38.594124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9c04f92a-0593-4e8c-a124-6d807cb58a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:38.614514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:38.702610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 570/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:38.996411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f286ba-0319-45d0-9da0-380012a847b8 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:38.998187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-09f286ba-0319-45d0-9da0-380012a847b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:39.120319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f286ba-0319-45d0-9da0-380012a847b8 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:39.200838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:39.210583 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f54ff1-09e8-421c-8ddd-f99728a0d520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:39.211441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f54ff1-09e8-421c-8ddd-f99728a0d520 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:39.211934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f54ff1-09e8-421c-8ddd-f99728a0d520 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:39.212750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f54ff1-09e8-421c-8ddd-f99728a0d520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:39.215549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-71f54ff1-09e8-421c-8ddd-f99728a0d520 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:39.365133 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f54ff1-09e8-421c-8ddd-f99728a0d520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:39.366619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f54ff1-09e8-421c-8ddd-f99728a0d520 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:39.391675 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 574/1144] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:34:41.701430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453e61-7c0a-4c9b-ab7f-0ef0899f4fd9 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:41.702041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453e61-7c0a-4c9b-ab7f-0ef0899f4fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:41.702374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09453e61-7c0a-4c9b-ab7f-0ef0899f4fd9 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:41.703947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 571/1145] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:34:41.710021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c42d8-a163-4fa2-b0e0-488331ef4fe6 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:41.710597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c42d8-a163-4fa2-b0e0-488331ef4fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:41.710597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c42d8-a163-4fa2-b0e0-488331ef4fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:41.786726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-1b4c42d8-a163-4fa2-b0e0-488331ef4fe6 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:34:41.787967 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-1b4c42d8-a163-4fa2-b0e0-488331ef4fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:41.845767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:41.908508 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 575/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:34:42.212259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453e61-7c0a-4c9b-ab7f-0ef0899f4fd9 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:34:42.213853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-09453e61-7c0a-4c9b-ab7f-0ef0899f4fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:42.355453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09453e61-7c0a-4c9b-ab7f-0ef0899f4fd9 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:34:42.413459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:42.442274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29519d-b9c5-4e4f-a0cd-e62a84a82878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:34:42.442806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29519d-b9c5-4e4f-a0cd-e62a84a82878 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:34:42.443088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29519d-b9c5-4e4f-a0cd-e62a84a82878 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:34:42.443361 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29519d-b9c5-4e4f-a0cd-e62a84a82878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:34:42.445031 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0a29519d-b9c5-4e4f-a0cd-e62a84a82878 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:34:42.604531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29519d-b9c5-4e4f-a0cd-e62a84a82878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:34:42.606820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29519d-b9c5-4e4f-a0cd-e62a84a82878 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:34:42.638075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 572/1147] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:34:51.906872 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=22 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:34:52.632875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=66 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:03.126482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd1ae8-f9e8-4bb2-aa1c-849df8f9047c None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:03.127052 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd1ae8-f9e8-4bb2-aa1c-849df8f9047c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:03.127052 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd1ae8-f9e8-4bb2-aa1c-849df8f9047c None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:03.128383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 576/1148] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:35:03.134547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b423fb-f88c-4b99-a833-c25987777ffd None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:03.135066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b423fb-f88c-4b99-a833-c25987777ffd None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:03.135066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b423fb-f88c-4b99-a833-c25987777ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:03.209868 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-99b423fb-f88c-4b99-a833-c25987777ffd None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:35:03.211083 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-99b423fb-f88c-4b99-a833-c25987777ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:03.314203 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 573/1149] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:35:03.622468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada83bd1-5262-4e14-9a8a-35dfff2dbbac None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:03.622746 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada83bd1-5262-4e14-9a8a-35dfff2dbbac None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:03.623014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada83bd1-5262-4e14-9a8a-35dfff2dbbac None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:03.624329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 577/1150] 10.222.0.17 () {54 vars in 962 bytes} [Tue Oct 18 11:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:35:03.629704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08df58-9012-4230-9e94-ff99e1c96779 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:03.630059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08df58-9012-4230-9e94-ff99e1c96779 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:03.630398 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08df58-9012-4230-9e94-ff99e1c96779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:03.713983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1c08df58-9012-4230-9e94-ff99e1c96779 None None] MFA Rules not processed for user `3d2a655f4d6f45039d641133e3b56592`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:35:03.715186 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1c08df58-9012-4230-9e94-ff99e1c96779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:03.735894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:03.793643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 574/1151] 10.222.0.17 () {58 vars in 1059 bytes} [Tue Oct 18 11:35:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:35:03.801411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada83bd1-5262-4e14-9a8a-35dfff2dbbac None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:35:03.803024 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ada83bd1-5262-4e14-9a8a-35dfff2dbbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:03.907818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada83bd1-5262-4e14-9a8a-35dfff2dbbac None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:35:03.985782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605a3cf-34a9-4e74-9742-736479d209ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d2a655f4d6f45039d641133e3b56592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:35:03.986399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605a3cf-34a9-4e74-9742-736479d209ad service glance] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:03.986680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605a3cf-34a9-4e74-9742-736479d209ad service glance] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:03.986942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605a3cf-34a9-4e74-9742-736479d209ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:03.988626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6605a3cf-34a9-4e74-9742-736479d209ad service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:04.058653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:04.138710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605a3cf-34a9-4e74-9742-736479d209ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:35:04.140165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605a3cf-34a9-4e74-9742-736479d209ad service glance] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:35:04.159045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 578/1152] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:35:06.443075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdafe-0c80-478c-88c8-8185a0e52266 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:06.443603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdafe-0c80-478c-88c8-8185a0e52266 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:06.443603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdafe-0c80-478c-88c8-8185a0e52266 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:06.444895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 575/1153] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:35:06.450952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f498f1b-d539-4426-9a9e-89c8ca158f51 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:06.451373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f498f1b-d539-4426-9a9e-89c8ca158f51 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:06.451493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f498f1b-d539-4426-9a9e-89c8ca158f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:06.522332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3f498f1b-d539-4426-9a9e-89c8ca158f51 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:35:06.523537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3f498f1b-d539-4426-9a9e-89c8ca158f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:06.621021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 579/1154] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:35:06.894243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcdafe-0c80-478c-88c8-8185a0e52266 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:35:06.895762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-dbfcdafe-0c80-478c-88c8-8185a0e52266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:07.018677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcdafe-0c80-478c-88c8-8185a0e52266 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:35:07.068729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:07.113906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f55a34-cf07-461e-b0eb-e95fafb6f3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:35:07.114522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55a34-cf07-461e-b0eb-e95fafb6f3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:07.114802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55a34-cf07-461e-b0eb-e95fafb6f3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:07.115073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55a34-cf07-461e-b0eb-e95fafb6f3f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:07.115713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f55a34-cf07-461e-b0eb-e95fafb6f3f1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:35:07.116735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WARNING py.warnings [None req-27f55a34-cf07-461e-b0eb-e95fafb6f3f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:35:07.116735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: warnings.warn(msg) Oct 18 11:35:07.116735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: Oct 18 11:35:07.117904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f55a34-cf07-461e-b0eb-e95fafb6f3f1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:35:07.138715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 576/1155] 10.222.0.17 () {56 vars in 1108 bytes} [Tue Oct 18 11:35:06 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:35:09.313880 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d198dd-25a0-4eff-a85c-29a70a5dfb92 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:09.314430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d198dd-25a0-4eff-a85c-29a70a5dfb92 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:09.314430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d198dd-25a0-4eff-a85c-29a70a5dfb92 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:09.315867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 580/1156] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:35:09.322064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a770-347e-426c-bfb0-34fa824be4de None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:09.322516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a770-347e-426c-bfb0-34fa824be4de None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:09.322611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50a770-347e-426c-bfb0-34fa824be4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:09.396815 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-cd50a770-347e-426c-bfb0-34fa824be4de None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:35:09.398085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cd50a770-347e-426c-bfb0-34fa824be4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:09.495008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 577/1157] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:35:09.793795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d198dd-25a0-4eff-a85c-29a70a5dfb92 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:35:09.795216 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c5d198dd-25a0-4eff-a85c-29a70a5dfb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:09.805422 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:09.951646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d198dd-25a0-4eff-a85c-29a70a5dfb92 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:35:10.033697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:10.068016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd154b31-840c-4a0c-9862-93d11121a667 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:35:10.068632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154b31-840c-4a0c-9862-93d11121a667 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:10.068939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154b31-840c-4a0c-9862-93d11121a667 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:10.069214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd154b31-840c-4a0c-9862-93d11121a667 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:10.070847 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fd154b31-840c-4a0c-9862-93d11121a667 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:10.162491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd154b31-840c-4a0c-9862-93d11121a667 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:35:10.164135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd154b31-840c-4a0c-9862-93d11121a667 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:35:10.184706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 581/1158] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:35:12.540069 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360ee42-ab70-450c-8352-4f0661c509ec None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:12.541197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360ee42-ab70-450c-8352-4f0661c509ec None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:12.541197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360ee42-ab70-450c-8352-4f0661c509ec None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:12.542482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 578/1159] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:35:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:35:12.547883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e76e-2ef1-499e-8f80-a283633049ed None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:12.548284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e76e-2ef1-499e-8f80-a283633049ed None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:12.548429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e76e-2ef1-499e-8f80-a283633049ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:12.615866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-dce4e76e-2ef1-499e-8f80-a283633049ed None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:35:12.617123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-dce4e76e-2ef1-499e-8f80-a283633049ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:12.677294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:12.733638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 582/1160] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:35:13.030831 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360ee42-ab70-450c-8352-4f0661c509ec None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:35:13.032415 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9360ee42-ab70-450c-8352-4f0661c509ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:13.056466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:13.136450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360ee42-ab70-450c-8352-4f0661c509ec None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:35:13.210672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771f331-e7d2-4cca-8843-2ffd382bbb98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:35:13.211287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f331-e7d2-4cca-8843-2ffd382bbb98 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:13.211560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f331-e7d2-4cca-8843-2ffd382bbb98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:13.211834 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f331-e7d2-4cca-8843-2ffd382bbb98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:13.213449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8771f331-e7d2-4cca-8843-2ffd382bbb98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:13.286798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:13.323883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771f331-e7d2-4cca-8843-2ffd382bbb98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:35:13.325350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771f331-e7d2-4cca-8843-2ffd382bbb98 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:35:13.343165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 579/1161] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:35:15.558980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051929-58be-46a1-9858-24f7e893e3cc None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:15.559514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051929-58be-46a1-9858-24f7e893e3cc None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:15.559514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051929-58be-46a1-9858-24f7e893e3cc None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:15.560778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 583/1162] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:35:15.566607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50446385-b41d-4643-8341-724125a5128d None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:15.567035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50446385-b41d-4643-8341-724125a5128d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:15.567433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50446385-b41d-4643-8341-724125a5128d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:15.641367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-50446385-b41d-4643-8341-724125a5128d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:35:15.642585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-50446385-b41d-4643-8341-724125a5128d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:15.741322 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 580/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:35:16.033268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da051929-58be-46a1-9858-24f7e893e3cc None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:35:16.034848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-da051929-58be-46a1-9858-24f7e893e3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:16.144415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da051929-58be-46a1-9858-24f7e893e3cc None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:35:16.223364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:16.225049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c6314-8560-4693-9d39-f464c5f1a340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:35:16.225995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c6314-8560-4693-9d39-f464c5f1a340 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:16.226506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c6314-8560-4693-9d39-f464c5f1a340 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:16.226989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c6314-8560-4693-9d39-f464c5f1a340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:16.229404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0a1c6314-8560-4693-9d39-f464c5f1a340 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:16.343890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c6314-8560-4693-9d39-f464c5f1a340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:35:16.345285 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c6314-8560-4693-9d39-f464c5f1a340 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:35:16.363979 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 584/1164] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:35:18.632481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe5469-ef78-44f1-91a7-9eb3db1733f6 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:18.633134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe5469-ef78-44f1-91a7-9eb3db1733f6 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:18.633134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe5469-ef78-44f1-91a7-9eb3db1733f6 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:18.634904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 581/1165] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:35:18.640256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac757376-799b-45fa-9803-3f49905617d0 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:18.640675 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac757376-799b-45fa-9803-3f49905617d0 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:18.640804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac757376-799b-45fa-9803-3f49905617d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:18.706296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ac757376-799b-45fa-9803-3f49905617d0 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:35:18.707512 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ac757376-799b-45fa-9803-3f49905617d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:18.760305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:18.804153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 585/1166] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:35:19.102764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe5469-ef78-44f1-91a7-9eb3db1733f6 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:35:19.104258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c2fe5469-ef78-44f1-91a7-9eb3db1733f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:19.144258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:19.214747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe5469-ef78-44f1-91a7-9eb3db1733f6 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:35:19.287384 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c660de-8747-48c6-a107-d74ea564dab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:35:19.288106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c660de-8747-48c6-a107-d74ea564dab2 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:19.288388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c660de-8747-48c6-a107-d74ea564dab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:19.288691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c660de-8747-48c6-a107-d74ea564dab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:19.290819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-63c660de-8747-48c6-a107-d74ea564dab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:19.374204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:19.403430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c660de-8747-48c6-a107-d74ea564dab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:35:19.404899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c660de-8747-48c6-a107-d74ea564dab2 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:35:19.422830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 582/1167] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:35:21.770005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f0a0-58f8-43a4-aff7-557a17579f85 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:21.770549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f0a0-58f8-43a4-aff7-557a17579f85 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:21.770549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f0a0-58f8-43a4-aff7-557a17579f85 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:21.771898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 586/1168] 10.222.0.17 () {54 vars in 927 bytes} [Tue Oct 18 11:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:35:21.777673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501506c-df60-4a48-bc98-a4e8e03353ef None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:21.778106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501506c-df60-4a48-bc98-a4e8e03353ef None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:21.778212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501506c-df60-4a48-bc98-a4e8e03353ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:21.844361 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e501506c-df60-4a48-bc98-a4e8e03353ef None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:35:21.845586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e501506c-df60-4a48-bc98-a4e8e03353ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:21.943385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 583/1169] 10.222.0.17 () {58 vars in 1024 bytes} [Tue Oct 18 11:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:35:22.239928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f0a0-58f8-43a4-aff7-557a17579f85 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:35:22.241584 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8179f0a0-58f8-43a4-aff7-557a17579f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:22.346277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f0a0-58f8-43a4-aff7-557a17579f85 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:35:22.425502 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d15a7a-622a-4300-a62c-3f7c62eb0b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:35:22.426178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d15a7a-622a-4300-a62c-3f7c62eb0b8b service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:35:22.426455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d15a7a-622a-4300-a62c-3f7c62eb0b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:35:22.426726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d15a7a-622a-4300-a62c-3f7c62eb0b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:35:22.428510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f7d15a7a-622a-4300-a62c-3f7c62eb0b8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:35:22.437532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:22.541462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d15a7a-622a-4300-a62c-3f7c62eb0b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:35:22.542891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d15a7a-622a-4300-a62c-3f7c62eb0b8b service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:35:22.561618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 587/1170] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:35:31.941819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=82 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:35:32.559339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=50 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:45.557933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d155d3-a9de-4cfb-b884-9afd682bf719 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:45.558714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d155d3-a9de-4cfb-b884-9afd682bf719 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:45.558714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d155d3-a9de-4cfb-b884-9afd682bf719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:45.655953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-85d155d3-a9de-4cfb-b884-9afd682bf719 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:36:45.658098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-85d155d3-a9de-4cfb-b884-9afd682bf719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:45.793357 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 584/1171] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:36:45.801766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30b9b5-948e-40fa-940c-51f15daca610 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:45.803058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30b9b5-948e-40fa-940c-51f15daca610 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:45.803058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30b9b5-948e-40fa-940c-51f15daca610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:45.887900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-8c30b9b5-948e-40fa-940c-51f15daca610 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:36:45.889280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8c30b9b5-948e-40fa-940c-51f15daca610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:46.013600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 588/1172] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:36:46.018374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d155d3-a9de-4cfb-b884-9afd682bf719 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:36:46.019645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-85d155d3-a9de-4cfb-b884-9afd682bf719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:46.032005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d155d3-a9de-4cfb-b884-9afd682bf719 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:36:46.101682 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:46.152784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:36:46.153340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:46.153621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:46.153895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:46.154565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:36:46.154876 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:36:46.155926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WARNING py.warnings [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:36:46.155926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: warnings.warn(msg) Oct 18 11:36:46.155926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: Oct 18 11:36:46.156872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:36:46.172332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 585/1173] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:36:46.182445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30b9b5-948e-40fa-940c-51f15daca610 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:36:46.193259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30b9b5-948e-40fa-940c-51f15daca610 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:36:46.272059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:46.315372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127efa4-0653-41ed-80d3-6ec5fea6cf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:36:46.316073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127efa4-0653-41ed-80d3-6ec5fea6cf63 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:46.316392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127efa4-0653-41ed-80d3-6ec5fea6cf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:46.316705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127efa4-0653-41ed-80d3-6ec5fea6cf63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:46.317455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127efa4-0653-41ed-80d3-6ec5fea6cf63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:36:46.319298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WARNING py.warnings [None req-8127efa4-0653-41ed-80d3-6ec5fea6cf63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:36:46.319298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: warnings.warn(msg) Oct 18 11:36:46.319298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: Oct 18 11:36:46.321167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127efa4-0653-41ed-80d3-6ec5fea6cf63 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:36:46.396602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 589/1174] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:36:46 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:36:46.401384 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:36:46.403579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:46.443127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65383714-a904-485b-bb11-eef0c81a2300 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:36:46.551556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1085a3c-7021-4d32-8d83-1f2752074e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:36:46.552245 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1085a3c-7021-4d32-8d83-1f2752074e1d admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:46.552560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1085a3c-7021-4d32-8d83-1f2752074e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:46.552942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1085a3c-7021-4d32-8d83-1f2752074e1d admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:46.553647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1085a3c-7021-4d32-8d83-1f2752074e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:36:46.554814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WARNING py.warnings [None req-a1085a3c-7021-4d32-8d83-1f2752074e1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:36:46.554814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: warnings.warn(msg) Oct 18 11:36:46.554814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: Oct 18 11:36:46.556223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1085a3c-7021-4d32-8d83-1f2752074e1d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:36:46.604630 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 586/1175] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:36:46 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:36:46.609456 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127efa4-0653-41ed-80d3-6ec5fea6cf63 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:36:46.611072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8127efa4-0653-41ed-80d3-6ec5fea6cf63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:46.678742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127efa4-0653-41ed-80d3-6ec5fea6cf63 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:36:46.774837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:46.787199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551dd4c-24fc-4c0b-91ca-53ef0a7b0cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:36:46.787937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551dd4c-24fc-4c0b-91ca-53ef0a7b0cdd admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:46.788259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551dd4c-24fc-4c0b-91ca-53ef0a7b0cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:46.788574 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551dd4c-24fc-4c0b-91ca-53ef0a7b0cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:46.789363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551dd4c-24fc-4c0b-91ca-53ef0a7b0cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:36:46.791014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551dd4c-24fc-4c0b-91ca-53ef0a7b0cdd admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:36:46.805534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 590/1176] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:36:46.811359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1085a3c-7021-4d32-8d83-1f2752074e1d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:36:46.814944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a1085a3c-7021-4d32-8d83-1f2752074e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:46.827853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1085a3c-7021-4d32-8d83-1f2752074e1d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:36:46.859052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:46.937674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcd8cd-3da2-4324-b3c5-6c2e33d86c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:36:46.938494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcd8cd-3da2-4324-b3c5-6c2e33d86c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:46.938864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcd8cd-3da2-4324-b3c5-6c2e33d86c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:46.939234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcd8cd-3da2-4324-b3c5-6c2e33d86c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:46.939988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcd8cd-3da2-4324-b3c5-6c2e33d86c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:36:46.941755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcd8cd-3da2-4324-b3c5-6c2e33d86c12 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:36:46.955687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 587/1177] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:36:46.960195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551dd4c-24fc-4c0b-91ca-53ef0a7b0cdd admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:36:46.961794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f551dd4c-24fc-4c0b-91ca-53ef0a7b0cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:47.002830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551dd4c-24fc-4c0b-91ca-53ef0a7b0cdd admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:36:47.107932 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b530945-6570-44e0-9b99-3db0d984ff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:36:47.108653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530945-6570-44e0-9b99-3db0d984ff15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:47.109014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530945-6570-44e0-9b99-3db0d984ff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:47.109339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530945-6570-44e0-9b99-3db0d984ff15 admin admin] PATH_INFO: `/v3/projects/3106a36d282848b99da3f918dc20015d/users/2b9faae9e7214c94909d0ee8a6534e04/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:47.134683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b530945-6570-44e0-9b99-3db0d984ff15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3106a36d282848b99da3f918dc20015d, user_id=2b9faae9e7214c94909d0ee8a6534e04, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:36:47.137571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b530945-6570-44e0-9b99-3db0d984ff15 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:36:47.166916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 591/1178] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:36:46 2022] PUT /identity/v3/projects/3106a36d282848b99da3f918dc20015d/users/2b9faae9e7214c94909d0ee8a6534e04/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:36:47.179566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee07ffa-419e-451c-9876-3ff6313c4a15 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:47.179865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee07ffa-419e-451c-9876-3ff6313c4a15 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:47.180104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee07ffa-419e-451c-9876-3ff6313c4a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:47.248177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-3ee07ffa-419e-451c-9876-3ff6313c4a15 None None] MFA Rules not processed for user `2b9faae9e7214c94909d0ee8a6534e04`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:36:47.249327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3ee07ffa-419e-451c-9876-3ff6313c4a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:47.336532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 588/1179] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:36:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:36:47.351199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32618e-daf0-47ee-9947-ffd41d424c2b None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:47.351477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32618e-daf0-47ee-9947-ffd41d424c2b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:47.351753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32618e-daf0-47ee-9947-ffd41d424c2b None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:47.353164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 592/1180] 10.222.0.17 () {54 vars in 963 bytes} [Tue Oct 18 11:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:36:47.358242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae101bb-f89b-48a0-93c5-f69113c07ad4 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:47.358527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae101bb-f89b-48a0-93c5-f69113c07ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:47.358803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae101bb-f89b-48a0-93c5-f69113c07ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:47.391722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:47.439260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6ae101bb-f89b-48a0-93c5-f69113c07ad4 None None] MFA Rules not processed for user `bb051b00a16949798158bac1ac57d36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:36:47.440558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6ae101bb-f89b-48a0-93c5-f69113c07ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:47.523501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 589/1181] 10.222.0.17 () {58 vars in 1060 bytes} [Tue Oct 18 11:36:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:36:47.533658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32618e-daf0-47ee-9947-ffd41d424c2b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:36:47.535293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ad32618e-daf0-47ee-9947-ffd41d424c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:47.592237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:47.652947 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32618e-daf0-47ee-9947-ffd41d424c2b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:36:47.738959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98d1a6-f353-4901-a01d-3d1cdc11d553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:36:47.740101 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98d1a6-f353-4901-a01d-3d1cdc11d553 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:47.740375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98d1a6-f353-4901-a01d-3d1cdc11d553 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:47.740644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98d1a6-f353-4901-a01d-3d1cdc11d553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:47.742316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-be98d1a6-f353-4901-a01d-3d1cdc11d553 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:47.857061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98d1a6-f353-4901-a01d-3d1cdc11d553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:36:47.858425 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98d1a6-f353-4901-a01d-3d1cdc11d553 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:36:47.862241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:47.884554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 593/1182] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:36:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:36:47.943118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffa9a5-ce6f-4b3d-87f2-0a549f06a89a None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:47.943681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffa9a5-ce6f-4b3d-87f2-0a549f06a89a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:47.943681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffa9a5-ce6f-4b3d-87f2-0a549f06a89a None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:47.944926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 590/1183] 10.222.0.17 () {52 vars in 830 bytes} [Tue Oct 18 11:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:36:47.948962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98d1a6-f353-4901-a01d-3d1cdc11d553 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:36:47.950268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-be98d1a6-f353-4901-a01d-3d1cdc11d553 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:48.058117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98d1a6-f353-4901-a01d-3d1cdc11d553 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:36:48.159509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:36:48.160149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:48.160426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:48.160696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] PATH_INFO: `/v3/users/2b9faae9e7214c94909d0ee8a6534e04` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:48.175077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b9faae9e7214c94909d0ee8a6534e04)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:36:48.176133 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WARNING py.warnings [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:36:48.176133 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: warnings.warn(msg) Oct 18 11:36:48.176133 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: Oct 18 11:36:48.177941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:36:48.189770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:48.256404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b9faae9e7214c94909d0ee8a6534e04'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:36:48.271691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b9faae9e7214c94909d0ee8a6534e04'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:36:48.272242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b9faae9e7214c94909d0ee8a6534e04'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:36:48.299061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b9faae9e7214c94909d0ee8a6534e04'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:36:48.310692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b9faae9e7214c94909d0ee8a6534e04'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:36:48.318779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 594/1184] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:36:47 2022] DELETE /identity/v3/users/2b9faae9e7214c94909d0ee8a6534e04 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:36:48.323817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1ae90-58ff-4d43-9cf3-1d0f2a0cdf14 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:48.324266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1ae90-58ff-4d43-9cf3-1d0f2a0cdf14 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:48.324633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1ae90-58ff-4d43-9cf3-1d0f2a0cdf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:48.395966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-34f1ae90-58ff-4d43-9cf3-1d0f2a0cdf14 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:36:48.397316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-34f1ae90-58ff-4d43-9cf3-1d0f2a0cdf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:48.483898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:48.497838 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 591/1185] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:36:48.508896 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:36:48.510317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:48.627314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965e756-c672-4aa2-be35-b3aca3055077 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:36:48.701134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:48.725897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdead2a9-384f-4b41-8131-281e8b027afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:36:48.726573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdead2a9-384f-4b41-8131-281e8b027afa service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:48.726895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdead2a9-384f-4b41-8131-281e8b027afa service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:48.727207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdead2a9-384f-4b41-8131-281e8b027afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:48.729124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fdead2a9-384f-4b41-8131-281e8b027afa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:48.880274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdead2a9-384f-4b41-8131-281e8b027afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:36:48.883018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdead2a9-384f-4b41-8131-281e8b027afa service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:36:48.918085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 595/1186] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:36:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:36:49.463219 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1ae90-58ff-4d43-9cf3-1d0f2a0cdf14 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:36:49.464560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-34f1ae90-58ff-4d43-9cf3-1d0f2a0cdf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:36:49.476711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1ae90-58ff-4d43-9cf3-1d0f2a0cdf14 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:36:49.601367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:36:49.602001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:36:49.602274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:36:49.602543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] PATH_INFO: `/v3/projects/3106a36d282848b99da3f918dc20015d` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:36:49.613366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3106a36d282848b99da3f918dc20015d)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:36:49.614453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WARNING py.warnings [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:36:49.614453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: warnings.warn(msg) Oct 18 11:36:49.614453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: Oct 18 11:36:49.615657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:36:49.683028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3106a36d282848b99da3f918dc20015d', 'request_id': 'req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:36:49.690639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3106a36d282848b99da3f918dc20015d', 'request_id': 'req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:36:49.704163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3106a36d282848b99da3f918dc20015d', 'request_id': 'req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:36:49.740663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] The token cache is being invalidate because project 3106a36d282848b99da3f918dc20015d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:36:49.740974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:36:49.741516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-c1a4c1fc-30d5-4d98-9cfc-e3f7ef170497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:36:49.742855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 592/1187] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:36:49 2022] DELETE /identity/v3/projects/3106a36d282848b99da3f918dc20015d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:36:58.913588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=62 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:36:59.738114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:38:19.554443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb342e7b-efc3-46e3-a578-f6c802017c2b None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:19.555008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb342e7b-efc3-46e3-a578-f6c802017c2b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:19.555008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb342e7b-efc3-46e3-a578-f6c802017c2b None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:19.556237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 596/1188] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:38:19.561240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ffceb-253f-4625-ba0d-d7ba6b1b27d0 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:19.561824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ffceb-253f-4625-ba0d-d7ba6b1b27d0 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:19.561824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ffceb-253f-4625-ba0d-d7ba6b1b27d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:19.646309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-960ffceb-253f-4625-ba0d-d7ba6b1b27d0 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:38:19.647560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-960ffceb-253f-4625-ba0d-d7ba6b1b27d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:19.752147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 593/1189] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:38:19.907851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb342e7b-efc3-46e3-a578-f6c802017c2b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:38:19.911003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-eb342e7b-efc3-46e3-a578-f6c802017c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:20.033996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb342e7b-efc3-46e3-a578-f6c802017c2b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:38:20.039416 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b80db-833d-4358-9e89-b01db9cc51ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:38:20.040015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b80db-833d-4358-9e89-b01db9cc51ee service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:20.040290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b80db-833d-4358-9e89-b01db9cc51ee service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:20.040558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b80db-833d-4358-9e89-b01db9cc51ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:20.049480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b80db-833d-4358-9e89-b01db9cc51ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:38:20.051059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b80db-833d-4358-9e89-b01db9cc51ee service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:38:20.060519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 597/1190] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:38:21.157416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbfaf7-5784-4e0f-93fb-707c20359915 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:21.158602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbfaf7-5784-4e0f-93fb-707c20359915 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:21.158602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbfaf7-5784-4e0f-93fb-707c20359915 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:21.159828 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 594/1191] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:38:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:38:21.166509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea81aac-9670-4f2e-ad01-d0001f2f0994 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:21.167070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea81aac-9670-4f2e-ad01-d0001f2f0994 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:21.167070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea81aac-9670-4f2e-ad01-d0001f2f0994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:21.187183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-cea81aac-9670-4f2e-ad01-d0001f2f0994 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:38:21.188389 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cea81aac-9670-4f2e-ad01-d0001f2f0994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:21.196742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 598/1192] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:38:21.349836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbfaf7-5784-4e0f-93fb-707c20359915 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:38:21.364712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbfaf7-5784-4e0f-93fb-707c20359915 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:38:21.370182 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f221f-f845-40b7-966e-e4b2e29721d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:38:21.370789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f221f-f845-40b7-966e-e4b2e29721d6 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:21.371156 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f221f-f845-40b7-966e-e4b2e29721d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:21.371329 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f221f-f845-40b7-966e-e4b2e29721d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:21.379824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f221f-f845-40b7-966e-e4b2e29721d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:38:21.381457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f221f-f845-40b7-966e-e4b2e29721d6 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:38:21.391376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 595/1193] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:38:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:38:21.584993 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee1ce-59e8-4fe7-aead-77187495cbaa None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:21.585281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee1ce-59e8-4fe7-aead-77187495cbaa None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:21.585558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee1ce-59e8-4fe7-aead-77187495cbaa None None] PATH_INFO: `/v3` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:21.586841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 599/1194] 10.222.0.42 () {54 vars in 925 bytes} [Tue Oct 18 11:38:21 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:38:21.594067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febce8ff-b395-4dd0-a2e9-2d9625a8a69e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:21.594352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febce8ff-b395-4dd0-a2e9-2d9625a8a69e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:21.594624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febce8ff-b395-4dd0-a2e9-2d9625a8a69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:21.662407 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-febce8ff-b395-4dd0-a2e9-2d9625a8a69e None None] MFA Rules not processed for user `99ec4a4276f345c2998723e554ab9608`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:38:21.663692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-febce8ff-b395-4dd0-a2e9-2d9625a8a69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:21.682126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:38:21.744055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 596/1195] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Oct 18 11:38:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:38:25.457078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c29c1-e07d-41ac-91d9-45caccd94b6e None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:25.457590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c29c1-e07d-41ac-91d9-45caccd94b6e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:25.457590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c29c1-e07d-41ac-91d9-45caccd94b6e None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:25.458943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 600/1196] 10.222.0.17 () {54 vars in 961 bytes} [Tue Oct 18 11:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:38:25.464181 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63027d9-cc5f-4e21-b362-3742dfc617a7 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:25.464652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63027d9-cc5f-4e21-b362-3742dfc617a7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:25.464746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63027d9-cc5f-4e21-b362-3742dfc617a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:25.490479 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-a63027d9-cc5f-4e21-b362-3742dfc617a7 None None] MFA Rules not processed for user `99ec4a4276f345c2998723e554ab9608`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:38:25.491692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a63027d9-cc5f-4e21-b362-3742dfc617a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:25.500352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 597/1197] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Oct 18 11:38:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:38:25.508779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c29c1-e07d-41ac-91d9-45caccd94b6e None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:38:25.522584 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c29c1-e07d-41ac-91d9-45caccd94b6e None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:38:25.527594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4883456-877b-4048-9242-251ff0094b51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:38:25.528196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4883456-877b-4048-9242-251ff0094b51 service placement] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:25.528475 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4883456-877b-4048-9242-251ff0094b51 service placement] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:25.528738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4883456-877b-4048-9242-251ff0094b51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:25.537528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4883456-877b-4048-9242-251ff0094b51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:38:25.539149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4883456-877b-4048-9242-251ff0094b51 service placement] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:38:25.549174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 601/1198] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:38:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:38:32.884320 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1cbdd-2df6-4603-9acb-6cc69a63c359 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:32.884953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1cbdd-2df6-4603-9acb-6cc69a63c359 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:32.884953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1cbdd-2df6-4603-9acb-6cc69a63c359 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:32.886276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 598/1199] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:38:32.893213 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce0973-b1e3-4f46-bb32-c9612a2309b9 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:32.893699 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce0973-b1e3-4f46-bb32-c9612a2309b9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:32.893801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce0973-b1e3-4f46-bb32-c9612a2309b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:32.913195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-83ce0973-b1e3-4f46-bb32-c9612a2309b9 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:38:32.914401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-83ce0973-b1e3-4f46-bb32-c9612a2309b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:32.923417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 602/1200] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:38:33.078130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1cbdd-2df6-4603-9acb-6cc69a63c359 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:38:33.093346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1cbdd-2df6-4603-9acb-6cc69a63c359 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:38:33.099069 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd894a7-f08d-424d-9493-9f4b915e4b5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:38:33.099672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd894a7-f08d-424d-9493-9f4b915e4b5f service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:33.099945 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd894a7-f08d-424d-9493-9f4b915e4b5f service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:33.100212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd894a7-f08d-424d-9493-9f4b915e4b5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:33.108477 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd894a7-f08d-424d-9493-9f4b915e4b5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:38:33.110120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd894a7-f08d-424d-9493-9f4b915e4b5f service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:38:33.119381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 599/1201] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:38:34.199559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637644e7-d9a8-4c2d-80fa-b5f644e2c6dc None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:34.200100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637644e7-d9a8-4c2d-80fa-b5f644e2c6dc None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:34.200100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637644e7-d9a8-4c2d-80fa-b5f644e2c6dc None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:34.201490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 603/1202] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 18 11:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:38:34.211136 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b8266-a8b3-404b-8f77-8d1bb38c3b00 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:34.211649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b8266-a8b3-404b-8f77-8d1bb38c3b00 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:34.211765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b8266-a8b3-404b-8f77-8d1bb38c3b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:34.232114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-950b8266-a8b3-404b-8f77-8d1bb38c3b00 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:38:34.233493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-950b8266-a8b3-404b-8f77-8d1bb38c3b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:34.241702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 600/1203] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 18 11:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:38:34.396494 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637644e7-d9a8-4c2d-80fa-b5f644e2c6dc None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:38:34.411870 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637644e7-d9a8-4c2d-80fa-b5f644e2c6dc None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:38:34.417455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2978f-42bb-4036-9533-8ee4d863cc9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:38:34.418055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2978f-42bb-4036-9533-8ee4d863cc9f service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:34.418331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2978f-42bb-4036-9533-8ee4d863cc9f service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:34.418601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2978f-42bb-4036-9533-8ee4d863cc9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:34.427373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2978f-42bb-4036-9533-8ee4d863cc9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:38:34.428985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2978f-42bb-4036-9533-8ee4d863cc9f service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:38:34.438438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 604/1204] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:38:38.525157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d99f9-9578-4d38-b06c-77fe86e1a8fa None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:38.525670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d99f9-9578-4d38-b06c-77fe86e1a8fa None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:38.525670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d99f9-9578-4d38-b06c-77fe86e1a8fa None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:38.526949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 601/1205] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:38:38.532377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a927-96ce-4203-b773-0d396c0dcf05 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:38.532873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a927-96ce-4203-b773-0d396c0dcf05 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:38.532989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a927-96ce-4203-b773-0d396c0dcf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:38.554666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e836a927-96ce-4203-b773-0d396c0dcf05 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:38:38.555885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e836a927-96ce-4203-b773-0d396c0dcf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:38.564609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 605/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:38:38.852864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d99f9-9578-4d38-b06c-77fe86e1a8fa None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:38:38.854628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4e3d99f9-9578-4d38-b06c-77fe86e1a8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:38.931130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d99f9-9578-4d38-b06c-77fe86e1a8fa None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:38:38.934682 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dcd5b-c27c-4320-b305-404d605912a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:38:38.935281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dcd5b-c27c-4320-b305-404d605912a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:38.935557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dcd5b-c27c-4320-b305-404d605912a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:38.935828 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dcd5b-c27c-4320-b305-404d605912a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:38.945206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dcd5b-c27c-4320-b305-404d605912a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:38:38.946621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dcd5b-c27c-4320-b305-404d605912a9 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:38:38.956383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 602/1207] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:38:41.135109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1744cc4-a3b5-4133-baa6-f89a2bc80584 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:41.135674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1744cc4-a3b5-4133-baa6-f89a2bc80584 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:41.135674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1744cc4-a3b5-4133-baa6-f89a2bc80584 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:41.137050 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 606/1208] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:38:41.142963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe24592-8db6-4285-a35d-ac8b09d55509 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:41.143411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe24592-8db6-4285-a35d-ac8b09d55509 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:41.143519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe24592-8db6-4285-a35d-ac8b09d55509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:41.163286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-dbe24592-8db6-4285-a35d-ac8b09d55509 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:38:41.164501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-dbe24592-8db6-4285-a35d-ac8b09d55509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:41.172630 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 603/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:38:41.461297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1744cc4-a3b5-4133-baa6-f89a2bc80584 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:38:41.472214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1744cc4-a3b5-4133-baa6-f89a2bc80584 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:38:41.475727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09071a3-a101-4882-903c-b2075cdbba25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:38:41.476314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09071a3-a101-4882-903c-b2075cdbba25 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:41.476586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09071a3-a101-4882-903c-b2075cdbba25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:41.476871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09071a3-a101-4882-903c-b2075cdbba25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:41.486470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09071a3-a101-4882-903c-b2075cdbba25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:38:41.487893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09071a3-a101-4882-903c-b2075cdbba25 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:38:41.498516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 607/1210] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:38:45.078723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1886f3d-8790-477b-b69c-b4ec8255dbb3 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:45.079832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1886f3d-8790-477b-b69c-b4ec8255dbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:45.079832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1886f3d-8790-477b-b69c-b4ec8255dbb3 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:45.081217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 604/1211] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:38:45.086664 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dcffe-5172-4cfa-977a-e1c229f473e2 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:45.087196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dcffe-5172-4cfa-977a-e1c229f473e2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:45.087327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dcffe-5172-4cfa-977a-e1c229f473e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:45.106943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-f52dcffe-5172-4cfa-977a-e1c229f473e2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:38:45.108154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f52dcffe-5172-4cfa-977a-e1c229f473e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:45.116059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 608/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:38:45.426390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1886f3d-8790-477b-b69c-b4ec8255dbb3 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:38:45.428071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d1886f3d-8790-477b-b69c-b4ec8255dbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:45.494214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:38:45.508875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1886f3d-8790-477b-b69c-b4ec8255dbb3 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:38:45.512447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49958d8-6c8c-4161-9913-7bbcf218a544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:38:45.513087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49958d8-6c8c-4161-9913-7bbcf218a544 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:45.513367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49958d8-6c8c-4161-9913-7bbcf218a544 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:45.513638 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49958d8-6c8c-4161-9913-7bbcf218a544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:45.522910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49958d8-6c8c-4161-9913-7bbcf218a544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:38:45.524353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49958d8-6c8c-4161-9913-7bbcf218a544 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:38:45.534436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 605/1213] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:38:47.747419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09570a8-41a2-4ad3-a117-60c816793ee9 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:47.748082 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09570a8-41a2-4ad3-a117-60c816793ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:47.748082 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09570a8-41a2-4ad3-a117-60c816793ee9 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:47.749448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 609/1214] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 18 11:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:38:47.755128 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73965d15-056f-4619-9a0b-37ebbbed864f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:47.755597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73965d15-056f-4619-9a0b-37ebbbed864f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:47.755701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73965d15-056f-4619-9a0b-37ebbbed864f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:47.777785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-73965d15-056f-4619-9a0b-37ebbbed864f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:38:47.779008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-73965d15-056f-4619-9a0b-37ebbbed864f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:47.787642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 606/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 18 11:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:38:48.088451 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09570a8-41a2-4ad3-a117-60c816793ee9 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:38:48.090193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f09570a8-41a2-4ad3-a117-60c816793ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:38:48.103705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09570a8-41a2-4ad3-a117-60c816793ee9 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:38:48.107173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f75e56-d8fb-4325-8f0f-7ff69ec6f1b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:38:48.107762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f75e56-d8fb-4325-8f0f-7ff69ec6f1b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:38:48.108038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f75e56-d8fb-4325-8f0f-7ff69ec6f1b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:38:48.108300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f75e56-d8fb-4325-8f0f-7ff69ec6f1b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:38:48.117614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f75e56-d8fb-4325-8f0f-7ff69ec6f1b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:38:48.119018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f75e56-d8fb-4325-8f0f-7ff69ec6f1b4 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:38:48.128689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 610/1216] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:38:57.772303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=11 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:38:58.124136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:14.040085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f8162-9e4d-43aa-a351-a671fc4eebb5 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.040744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f8162-9e4d-43aa-a351-a671fc4eebb5 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.040744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f8162-9e4d-43aa-a351-a671fc4eebb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.065925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-5f2f8162-9e4d-43aa-a351-a671fc4eebb5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.067552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5f2f8162-9e4d-43aa-a351-a671fc4eebb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.082294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 607/1217] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.093914 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27902ca9-e661-4495-890f-f93ac7da7fe6 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.094519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27902ca9-e661-4495-890f-f93ac7da7fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.094612 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27902ca9-e661-4495-890f-f93ac7da7fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.126030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-27902ca9-e661-4495-890f-f93ac7da7fe6 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.127432 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-27902ca9-e661-4495-890f-f93ac7da7fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.136185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 611/1218] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.141675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f8162-9e4d-43aa-a351-a671fc4eebb5 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:14.157836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f8162-9e4d-43aa-a351-a671fc4eebb5 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:14.167747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998092b-90b2-4b99-af61-6a78a09dba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:14.169413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998092b-90b2-4b99-af61-6a78a09dba51 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.169968 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998092b-90b2-4b99-af61-6a78a09dba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.170456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998092b-90b2-4b99-af61-6a78a09dba51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.171338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998092b-90b2-4b99-af61-6a78a09dba51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:14.171893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998092b-90b2-4b99-af61-6a78a09dba51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:14.173958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998092b-90b2-4b99-af61-6a78a09dba51 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:14.194706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 608/1219] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:14.203846 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27902ca9-e661-4495-890f-f93ac7da7fe6 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:14.223213 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27902ca9-e661-4495-890f-f93ac7da7fe6 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:14.233821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49ad12-ee31-4002-b866-1920aae10f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:14.234862 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49ad12-ee31-4002-b866-1920aae10f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.235371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49ad12-ee31-4002-b866-1920aae10f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.235879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49ad12-ee31-4002-b866-1920aae10f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.236803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49ad12-ee31-4002-b866-1920aae10f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:14.239064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49ad12-ee31-4002-b866-1920aae10f1a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:14.295927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031c18c-6754-4ebd-a90f-ff738dcb9cab None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.296832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031c18c-6754-4ebd-a90f-ff738dcb9cab None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.297364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031c18c-6754-4ebd-a90f-ff738dcb9cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.325642 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 612/1220] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:14.330917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49ad12-ee31-4002-b866-1920aae10f1a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:14.344064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c031c18c-6754-4ebd-a90f-ff738dcb9cab None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.352210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c031c18c-6754-4ebd-a90f-ff738dcb9cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.495743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49ad12-ee31-4002-b866-1920aae10f1a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:14.512270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb15d15-9eaf-4dfd-bcaf-c107a61601c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:14.512927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb15d15-9eaf-4dfd-bcaf-c107a61601c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.513210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb15d15-9eaf-4dfd-bcaf-c107a61601c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.514667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb15d15-9eaf-4dfd-bcaf-c107a61601c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.516942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb15d15-9eaf-4dfd-bcaf-c107a61601c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:14.520487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WARNING py.warnings [None req-ceb15d15-9eaf-4dfd-bcaf-c107a61601c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:40:14.520487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: warnings.warn(msg) Oct 18 11:40:14.520487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: Oct 18 11:40:14.527822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 609/1221] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:40:14.527822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d07789-d158-40f2-9734-13562a029d9c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.528124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d07789-d158-40f2-9734-13562a029d9c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.528347 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d07789-d158-40f2-9734-13562a029d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.530976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb15d15-9eaf-4dfd-bcaf-c107a61601c5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:14.578369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-75d07789-d158-40f2-9734-13562a029d9c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.585681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-75d07789-d158-40f2-9734-13562a029d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.587458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 613/1222] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:40:14.593112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8759e-f1c7-4675-ab82-27e910196d2c None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.593529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8759e-f1c7-4675-ab82-27e910196d2c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.593818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8759e-f1c7-4675-ab82-27e910196d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.612078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 610/1223] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.614820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccec129-1f90-4c88-b8d4-9b32ddcb750e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.621350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccec129-1f90-4c88-b8d4-9b32ddcb750e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.621696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccec129-1f90-4c88-b8d4-9b32ddcb750e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.631837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-6ca8759e-f1c7-4675-ab82-27e910196d2c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.633081 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6ca8759e-f1c7-4675-ab82-27e910196d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.656315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 614/1224] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.660054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8759e-f1c7-4675-ab82-27e910196d2c None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:14.661423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1ccec129-1f90-4c88-b8d4-9b32ddcb750e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.662627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1ccec129-1f90-4c88-b8d4-9b32ddcb750e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.675405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8759e-f1c7-4675-ab82-27e910196d2c None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:14.678733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 611/1225] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.680248 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492ef1-6f92-451a-bafb-942538fb5127 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.680536 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492ef1-6f92-451a-bafb-942538fb5127 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.680801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02c340-ed40-494e-b0a0-5c32af8ba9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:14.680960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492ef1-6f92-451a-bafb-942538fb5127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.681555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02c340-ed40-494e-b0a0-5c32af8ba9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.681896 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02c340-ed40-494e-b0a0-5c32af8ba9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.682247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02c340-ed40-494e-b0a0-5c32af8ba9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.682958 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02c340-ed40-494e-b0a0-5c32af8ba9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:14.685318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02c340-ed40-494e-b0a0-5c32af8ba9d5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:14.703542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 615/1226] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:14.705317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679a651-e9e8-47f6-9df8-01a5d50e11ae None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.705839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679a651-e9e8-47f6-9df8-01a5d50e11ae None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.706692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679a651-e9e8-47f6-9df8-01a5d50e11ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.707153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-22492ef1-6f92-451a-bafb-942538fb5127 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.708375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-22492ef1-6f92-451a-bafb-942538fb5127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.721934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 612/1227] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.722917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22492ef1-6f92-451a-bafb-942538fb5127 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:14.731489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7679a651-e9e8-47f6-9df8-01a5d50e11ae None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.732853 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7679a651-e9e8-47f6-9df8-01a5d50e11ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.733228 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22492ef1-6f92-451a-bafb-942538fb5127 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:14.738481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba97d22-abda-4a32-a0e9-b8bc53a581b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:14.739100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba97d22-abda-4a32-a0e9-b8bc53a581b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.739377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba97d22-abda-4a32-a0e9-b8bc53a581b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.739647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba97d22-abda-4a32-a0e9-b8bc53a581b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.740307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba97d22-abda-4a32-a0e9-b8bc53a581b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:14.740609 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba97d22-abda-4a32-a0e9-b8bc53a581b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:14.742331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba97d22-abda-4a32-a0e9-b8bc53a581b4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:14.753472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 616/1228] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.755454 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72530870-566b-4de2-8a14-cb2b45cbfe73 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.755454 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72530870-566b-4de2-8a14-cb2b45cbfe73 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.755749 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72530870-566b-4de2-8a14-cb2b45cbfe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.756990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 613/1229] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:14.758567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523954a7-7a2b-4552-a929-02373a61fadd None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.759203 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523954a7-7a2b-4552-a929-02373a61fadd None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.759203 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523954a7-7a2b-4552-a929-02373a61fadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.779008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-72530870-566b-4de2-8a14-cb2b45cbfe73 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.780277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-72530870-566b-4de2-8a14-cb2b45cbfe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.781863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-523954a7-7a2b-4552-a929-02373a61fadd None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.783113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-523954a7-7a2b-4552-a929-02373a61fadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.788915 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 617/1230] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.789820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72530870-566b-4de2-8a14-cb2b45cbfe73 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:14.793230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 614/1231] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.794957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7ea84-aa57-430d-bd75-5b43685e66a5 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.795403 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7ea84-aa57-430d-bd75-5b43685e66a5 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.795861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7ea84-aa57-430d-bd75-5b43685e66a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.804951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72530870-566b-4de2-8a14-cb2b45cbfe73 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:14.809748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacd9f6-9033-4ad9-a61e-e569bceb1964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:14.810904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd9f6-9033-4ad9-a61e-e569bceb1964 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.811182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd9f6-9033-4ad9-a61e-e569bceb1964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.811452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd9f6-9033-4ad9-a61e-e569bceb1964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.812098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacd9f6-9033-4ad9-a61e-e569bceb1964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:14.813738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacd9f6-9033-4ad9-a61e-e569bceb1964 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:14.818774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-7ce7ea84-aa57-430d-bd75-5b43685e66a5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.819994 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7ce7ea84-aa57-430d-bd75-5b43685e66a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.824556 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 618/1232] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:14.825448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacd9f6-9033-4ad9-a61e-e569bceb1964 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:14.829450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 615/1233] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.830714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c55f2-f707-40eb-8d69-09bca626a489 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.830999 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c55f2-f707-40eb-8d69-09bca626a489 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.831272 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c55f2-f707-40eb-8d69-09bca626a489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.838706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacd9f6-9033-4ad9-a61e-e569bceb1964 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:14.843846 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad4a45-90df-4827-a6f6-38a5ffcfc546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:14.844466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad4a45-90df-4827-a6f6-38a5ffcfc546 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.844932 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad4a45-90df-4827-a6f6-38a5ffcfc546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.845209 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad4a45-90df-4827-a6f6-38a5ffcfc546 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.845867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad4a45-90df-4827-a6f6-38a5ffcfc546 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:14.846170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad4a45-90df-4827-a6f6-38a5ffcfc546 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:14.847214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WARNING py.warnings [None req-73ad4a45-90df-4827-a6f6-38a5ffcfc546 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:40:14.847214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: warnings.warn(msg) Oct 18 11:40:14.847214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: Oct 18 11:40:14.848104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad4a45-90df-4827-a6f6-38a5ffcfc546 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:14.854498 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-076c55f2-f707-40eb-8d69-09bca626a489 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.855711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-076c55f2-f707-40eb-8d69-09bca626a489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.860573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 619/1234] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:14.861441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad4a45-90df-4827-a6f6-38a5ffcfc546 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:14.868700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 616/1235] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.868700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0375f-878a-4c53-8485-5e834aa733ad None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.868700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0375f-878a-4c53-8485-5e834aa733ad None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.868700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0375f-878a-4c53-8485-5e834aa733ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.880884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad4a45-90df-4827-a6f6-38a5ffcfc546 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:14.885744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6b6ef-49c6-4a3a-b001-f6b454009929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:14.886455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6b6ef-49c6-4a3a-b001-f6b454009929 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.886735 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6b6ef-49c6-4a3a-b001-f6b454009929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.887028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6b6ef-49c6-4a3a-b001-f6b454009929 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.887708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6b6ef-49c6-4a3a-b001-f6b454009929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:14.889830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e7b0375f-878a-4c53-8485-5e834aa733ad None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.890265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6b6ef-49c6-4a3a-b001-f6b454009929 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:14.891028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e7b0375f-878a-4c53-8485-5e834aa733ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.899200 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 617/1236] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:14.900202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0375f-878a-4c53-8485-5e834aa733ad None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:14.911901 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0375f-878a-4c53-8485-5e834aa733ad None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:14.917864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d489862-3122-444b-990f-2f011c4b4bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:14.918770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d489862-3122-444b-990f-2f011c4b4bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.919049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d489862-3122-444b-990f-2f011c4b4bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.919317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d489862-3122-444b-990f-2f011c4b4bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.920016 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d489862-3122-444b-990f-2f011c4b4bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:14.920306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d489862-3122-444b-990f-2f011c4b4bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:14.921878 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d489862-3122-444b-990f-2f011c4b4bf4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:14.923089 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 620/1237] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/projects => generated 355 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:14.924366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737be19-508e-498a-97e9-63d0e303ff61 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:14.924654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737be19-508e-498a-97e9-63d0e303ff61 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:14.924954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737be19-508e-498a-97e9-63d0e303ff61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:14.934794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 618/1238] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:14.936077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d489862-3122-444b-990f-2f011c4b4bf4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:14.946427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d489862-3122-444b-990f-2f011c4b4bf4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:14.950168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4737be19-508e-498a-97e9-63d0e303ff61 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:14.952289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4737be19-508e-498a-97e9-63d0e303ff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:14.984462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:15.009321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92637f01-4a12-4c5f-aef4-c7eaa0d94a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.010014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92637f01-4a12-4c5f-aef4-c7eaa0d94a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.010414 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92637f01-4a12-4c5f-aef4-c7eaa0d94a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.010841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92637f01-4a12-4c5f-aef4-c7eaa0d94a31 admin admin] PATH_INFO: `/v3/projects/ffc96760658d4e19af26d6cf22e94172/users/bddd3d2bb8f647f2ba616141eb16622e/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.020006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 621/1239] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:15.021670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc76b-33f0-4c27-94d9-63cadf15c0cc None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.022344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc76b-33f0-4c27-94d9-63cadf15c0cc None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.022344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cc76b-33f0-4c27-94d9-63cadf15c0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.029851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:15.033097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92637f01-4a12-4c5f-aef4-c7eaa0d94a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc96760658d4e19af26d6cf22e94172, user_id=bddd3d2bb8f647f2ba616141eb16622e, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.036483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92637f01-4a12-4c5f-aef4-c7eaa0d94a31 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.045111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-8b7cc76b-33f0-4c27-94d9-63cadf15c0cc None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:15.046381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8b7cc76b-33f0-4c27-94d9-63cadf15c0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:15.051531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 619/1240] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:14 2022] PUT /identity/v3/projects/ffc96760658d4e19af26d6cf22e94172/users/bddd3d2bb8f647f2ba616141eb16622e/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:15.053007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92637f01-4a12-4c5f-aef4-c7eaa0d94a31 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.114965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 622/1241] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:15.116641 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cc76b-33f0-4c27-94d9-63cadf15c0cc None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.117965 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92637f01-4a12-4c5f-aef4-c7eaa0d94a31 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.123844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17c422-ae11-4200-a805-de452f3103f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.124082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17c422-ae11-4200-a805-de452f3103f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.124347 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17c422-ae11-4200-a805-de452f3103f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.124617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17c422-ae11-4200-a805-de452f3103f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.125301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17c422-ae11-4200-a805-de452f3103f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.126321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WARNING py.warnings [None req-eb17c422-ae11-4200-a805-de452f3103f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:40:15.126321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: warnings.warn(msg) Oct 18 11:40:15.126321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: Oct 18 11:40:15.127401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cc76b-33f0-4c27-94d9-63cadf15c0cc None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.127517 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17c422-ae11-4200-a805-de452f3103f2 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.132044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d1f02-9e88-456d-aeb4-b54cf25a5ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.132655 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d1f02-9e88-456d-aeb4-b54cf25a5ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.132955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d1f02-9e88-456d-aeb4-b54cf25a5ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.133225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d1f02-9e88-456d-aeb4-b54cf25a5ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.133915 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d1f02-9e88-456d-aeb4-b54cf25a5ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:15.134192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d1f02-9e88-456d-aeb4-b54cf25a5ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.135791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d1f02-9e88-456d-aeb4-b54cf25a5ab8 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.148288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 623/1242] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:15.149528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7b03a-3969-4867-8562-e89b01311e39 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.149880 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7b03a-3969-4867-8562-e89b01311e39 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.150154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7b03a-3969-4867-8562-e89b01311e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.172763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 620/1243] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/projects => generated 393 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.172763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17c422-ae11-4200-a805-de452f3103f2 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.173548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b6d7b03a-3969-4867-8562-e89b01311e39 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:15.174762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b6d7b03a-3969-4867-8562-e89b01311e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:15.240357 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17c422-ae11-4200-a805-de452f3103f2 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.245118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0298da4-8a3f-4762-b8b2-863991a6459f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.245736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0298da4-8a3f-4762-b8b2-863991a6459f admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.246154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0298da4-8a3f-4762-b8b2-863991a6459f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.246476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0298da4-8a3f-4762-b8b2-863991a6459f admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.247171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0298da4-8a3f-4762-b8b2-863991a6459f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.247287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 624/1244] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:15.248859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7b03a-3969-4867-8562-e89b01311e39 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.250039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0298da4-8a3f-4762-b8b2-863991a6459f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.258821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7b03a-3969-4867-8562-e89b01311e39 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.264098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a697bc4-0556-4204-8892-62d0553ca8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.264737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a697bc4-0556-4204-8892-62d0553ca8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.265029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a697bc4-0556-4204-8892-62d0553ca8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.265304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a697bc4-0556-4204-8892-62d0553ca8c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.265962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a697bc4-0556-4204-8892-62d0553ca8c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.268030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a697bc4-0556-4204-8892-62d0553ca8c7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.278231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 621/1245] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/users => generated 278 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.278231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0298da4-8a3f-4762-b8b2-863991a6459f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.287209 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0298da4-8a3f-4762-b8b2-863991a6459f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.292798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f18608-20f8-40c1-876f-1189a8a17850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.293473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f18608-20f8-40c1-876f-1189a8a17850 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.293754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f18608-20f8-40c1-876f-1189a8a17850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.294025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f18608-20f8-40c1-876f-1189a8a17850 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.295210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f18608-20f8-40c1-876f-1189a8a17850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:15.295514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f18608-20f8-40c1-876f-1189a8a17850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.297146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f18608-20f8-40c1-876f-1189a8a17850 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.300568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 625/1246] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.301934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3654501-ad65-4fdf-9460-40ef73588d88 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.302215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3654501-ad65-4fdf-9460-40ef73588d88 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.302490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3654501-ad65-4fdf-9460-40ef73588d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.312008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 622/1247] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:15.316563 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f18608-20f8-40c1-876f-1189a8a17850 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.322870 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e3654501-ad65-4fdf-9460-40ef73588d88 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:15.323493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f18608-20f8-40c1-876f-1189a8a17850 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.324104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e3654501-ad65-4fdf-9460-40ef73588d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:15.367095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:15.380948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:15.391123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a4f81-056b-4669-85c7-5985b121004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.391756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4f81-056b-4669-85c7-5985b121004c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.392036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4f81-056b-4669-85c7-5985b121004c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.392333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4f81-056b-4669-85c7-5985b121004c admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.395768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 626/1248] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:15.395915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a4f81-056b-4669-85c7-5985b121004c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:15.395915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a4f81-056b-4669-85c7-5985b121004c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.396110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3654501-ad65-4fdf-9460-40ef73588d88 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.396216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a4f81-056b-4669-85c7-5985b121004c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.406415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3654501-ad65-4fdf-9460-40ef73588d88 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.415019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 623/1249] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:15.415019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a4f81-056b-4669-85c7-5985b121004c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.415411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df5b51-a727-4b9d-aba8-5de27bf1afbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.415411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df5b51-a727-4b9d-aba8-5de27bf1afbe admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.415411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df5b51-a727-4b9d-aba8-5de27bf1afbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.415411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df5b51-a727-4b9d-aba8-5de27bf1afbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.415787 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df5b51-a727-4b9d-aba8-5de27bf1afbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.416507 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df5b51-a727-4b9d-aba8-5de27bf1afbe admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.423180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a4f81-056b-4669-85c7-5985b121004c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.428082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1b604-4197-4294-a161-cafd3e004889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.428705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1b604-4197-4294-a161-cafd3e004889 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.429018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1b604-4197-4294-a161-cafd3e004889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.429290 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1b604-4197-4294-a161-cafd3e004889 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.429962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1b604-4197-4294-a161-cafd3e004889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.431894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1b604-4197-4294-a161-cafd3e004889 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.447571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 627/1250] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.448275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df5b51-a727-4b9d-aba8-5de27bf1afbe admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.455363 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 624/1251] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.456301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1b604-4197-4294-a161-cafd3e004889 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.466613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1b604-4197-4294-a161-cafd3e004889 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.519841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df5b51-a727-4b9d-aba8-5de27bf1afbe admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.529834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0b36d-4f76-403e-af3e-10231b8abd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.530457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b36d-4f76-403e-af3e-10231b8abd91 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.530584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9e793-c5dc-416d-a872-31374df7af7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.530729 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b36d-4f76-403e-af3e-10231b8abd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.530995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b36d-4f76-403e-af3e-10231b8abd91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.531319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9e793-c5dc-416d-a872-31374df7af7f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.531639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9e793-c5dc-416d-a872-31374df7af7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.531747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0b36d-4f76-403e-af3e-10231b8abd91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:15.531914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9e793-c5dc-416d-a872-31374df7af7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.532016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0b36d-4f76-403e-af3e-10231b8abd91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.532582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9e793-c5dc-416d-a872-31374df7af7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.533692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0b36d-4f76-403e-af3e-10231b8abd91 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.534257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9e793-c5dc-416d-a872-31374df7af7f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.545367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 625/1252] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:15.546204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9e793-c5dc-416d-a872-31374df7af7f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.546843 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 628/1253] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:15.547719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0b36d-4f76-403e-af3e-10231b8abd91 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.556756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9e793-c5dc-416d-a872-31374df7af7f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.558412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0b36d-4f76-403e-af3e-10231b8abd91 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.563357 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e43ed-8823-40a9-a2d5-b5fdb0286499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.563973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e43ed-8823-40a9-a2d5-b5fdb0286499 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.564203 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4205d8-fcf7-46be-b93f-1a02156608f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.564332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e43ed-8823-40a9-a2d5-b5fdb0286499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.564507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e43ed-8823-40a9-a2d5-b5fdb0286499 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.564818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4205d8-fcf7-46be-b93f-1a02156608f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.565137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4205d8-fcf7-46be-b93f-1a02156608f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.565241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e43ed-8823-40a9-a2d5-b5fdb0286499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.565408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4205d8-fcf7-46be-b93f-1a02156608f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.566046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4205d8-fcf7-46be-b93f-1a02156608f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.567081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e43ed-8823-40a9-a2d5-b5fdb0286499 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.567954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4205d8-fcf7-46be-b93f-1a02156608f3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.591007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 626/1254] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.595886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e43ed-8823-40a9-a2d5-b5fdb0286499 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.602657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 629/1255] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.603773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4205d8-fcf7-46be-b93f-1a02156608f3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.657269 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e43ed-8823-40a9-a2d5-b5fdb0286499 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.669794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4205d8-fcf7-46be-b93f-1a02156608f3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.674451 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63722059-1f8e-4392-92bd-cc40213b3df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.675071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63722059-1f8e-4392-92bd-cc40213b3df7 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.675350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63722059-1f8e-4392-92bd-cc40213b3df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.675614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63722059-1f8e-4392-92bd-cc40213b3df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.676244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63722059-1f8e-4392-92bd-cc40213b3df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.678240 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63722059-1f8e-4392-92bd-cc40213b3df7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.711343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 630/1256] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.711556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b7ed8-3e1d-4625-8dca-0ffc45421f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.711703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63722059-1f8e-4392-92bd-cc40213b3df7 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.711799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b7ed8-3e1d-4625-8dca-0ffc45421f98 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.711799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b7ed8-3e1d-4625-8dca-0ffc45421f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.711799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b7ed8-3e1d-4625-8dca-0ffc45421f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.731484 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b7ed8-3e1d-4625-8dca-0ffc45421f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.733024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b7ed8-3e1d-4625-8dca-0ffc45421f98 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.733482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:15.743593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 627/1257] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:15.744519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b7ed8-3e1d-4625-8dca-0ffc45421f98 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.755383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b7ed8-3e1d-4625-8dca-0ffc45421f98 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.759916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:15.787144 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63722059-1f8e-4392-92bd-cc40213b3df7 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.795003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105e130-a9fc-44cb-be93-74e4cde1b2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.795737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105e130-a9fc-44cb-be93-74e4cde1b2fb admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.796080 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105e130-a9fc-44cb-be93-74e4cde1b2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.796419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105e130-a9fc-44cb-be93-74e4cde1b2fb admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.797168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105e130-a9fc-44cb-be93-74e4cde1b2fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.799245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105e130-a9fc-44cb-be93-74e4cde1b2fb admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.825315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 631/1258] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.826545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105e130-a9fc-44cb-be93-74e4cde1b2fb admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.833459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2cbc8-db2e-4f93-a1a4-ff9a7d18f031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.834164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cbc8-db2e-4f93-a1a4-ff9a7d18f031 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.834614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cbc8-db2e-4f93-a1a4-ff9a7d18f031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.835014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cbc8-db2e-4f93-a1a4-ff9a7d18f031 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.835753 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2cbc8-db2e-4f93-a1a4-ff9a7d18f031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.837277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105e130-a9fc-44cb-be93-74e4cde1b2fb admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.837512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2cbc8-db2e-4f93-a1a4-ff9a7d18f031 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.842985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a7123-e4bb-4776-9bec-0a79393c67de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.843840 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a7123-e4bb-4776-9bec-0a79393c67de admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.843975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a7123-e4bb-4776-9bec-0a79393c67de admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.845106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a7123-e4bb-4776-9bec-0a79393c67de admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.845524 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a7123-e4bb-4776-9bec-0a79393c67de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.847124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a7123-e4bb-4776-9bec-0a79393c67de admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.852433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 628/1259] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:15.853424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2cbc8-db2e-4f93-a1a4-ff9a7d18f031 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.859877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 632/1260] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:15.861001 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a7123-e4bb-4776-9bec-0a79393c67de admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.867934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2cbc8-db2e-4f93-a1a4-ff9a7d18f031 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.871404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a7123-e4bb-4776-9bec-0a79393c67de admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.876693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fa597-150f-4073-8634-1b6ab69f4e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.877014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038dabf-c22f-48a8-bac2-28e43ae8c2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.877374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fa597-150f-4073-8634-1b6ab69f4e1e admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.877609 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038dabf-c22f-48a8-bac2-28e43ae8c2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.877715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fa597-150f-4073-8634-1b6ab69f4e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.877898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038dabf-c22f-48a8-bac2-28e43ae8c2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.878018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fa597-150f-4073-8634-1b6ab69f4e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.878213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038dabf-c22f-48a8-bac2-28e43ae8c2a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.878539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fa597-150f-4073-8634-1b6ab69f4e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.878861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038dabf-c22f-48a8-bac2-28e43ae8c2a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.880155 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fa597-150f-4073-8634-1b6ab69f4e1e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.880936 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038dabf-c22f-48a8-bac2-28e43ae8c2a9 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.892413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 633/1261] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:15.893505 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fa597-150f-4073-8634-1b6ab69f4e1e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.905188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fa597-150f-4073-8634-1b6ab69f4e1e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:15.914249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22effc7-5714-4346-a1b6-c4efdc8e76f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:15.915331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22effc7-5714-4346-a1b6-c4efdc8e76f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:15.915829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22effc7-5714-4346-a1b6-c4efdc8e76f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:15.916340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22effc7-5714-4346-a1b6-c4efdc8e76f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:15.917656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22effc7-5714-4346-a1b6-c4efdc8e76f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:15.921012 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 629/1262] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.922188 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038dabf-c22f-48a8-bac2-28e43ae8c2a9 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.922458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22effc7-5714-4346-a1b6-c4efdc8e76f3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:15.956498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 634/1263] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/users => generated 302 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:15.956498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22effc7-5714-4346-a1b6-c4efdc8e76f3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:15.969566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22effc7-5714-4346-a1b6-c4efdc8e76f3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.020212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038dabf-c22f-48a8-bac2-28e43ae8c2a9 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.025866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f2d76-a31a-460e-8682-af95b730223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.026588 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f2d76-a31a-460e-8682-af95b730223d admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.026882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f2d76-a31a-460e-8682-af95b730223d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.027202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f2d76-a31a-460e-8682-af95b730223d admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.028007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f2d76-a31a-460e-8682-af95b730223d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.030305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f2d76-a31a-460e-8682-af95b730223d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.047059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badda1d9-5d58-486d-b99d-03c201f6c95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.048072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badda1d9-5d58-486d-b99d-03c201f6c95f admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.048365 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badda1d9-5d58-486d-b99d-03c201f6c95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.048744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badda1d9-5d58-486d-b99d-03c201f6c95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.049804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badda1d9-5d58-486d-b99d-03c201f6c95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.051784 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badda1d9-5d58-486d-b99d-03c201f6c95f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.060109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 630/1264] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:15 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:16.061973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f2d76-a31a-460e-8682-af95b730223d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.065359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 635/1265] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:16.066482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badda1d9-5d58-486d-b99d-03c201f6c95f admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.074539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f2d76-a31a-460e-8682-af95b730223d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.080737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badda1d9-5d58-486d-b99d-03c201f6c95f admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.084635 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb69bf8-2190-45cc-8d3e-d3d86c7a6fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.085729 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ead484-fc60-413b-87dd-1ccb764259fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.085892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb69bf8-2190-45cc-8d3e-d3d86c7a6fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.086236 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb69bf8-2190-45cc-8d3e-d3d86c7a6fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.086523 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ead484-fc60-413b-87dd-1ccb764259fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.086685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb69bf8-2190-45cc-8d3e-d3d86c7a6fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.086879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ead484-fc60-413b-87dd-1ccb764259fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.087257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ead484-fc60-413b-87dd-1ccb764259fb admin admin] PATH_INFO: `/v3/projects/15fafabc29464e789bc4fa83f3389a20/users/b04ad85928b24b969e923ca20b2a453a/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.087910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb69bf8-2190-45cc-8d3e-d3d86c7a6fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.091191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb69bf8-2190-45cc-8d3e-d3d86c7a6fb5 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.108602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 631/1266] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:16.109513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb69bf8-2190-45cc-8d3e-d3d86c7a6fb5 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.118384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ead484-fc60-413b-87dd-1ccb764259fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=15fafabc29464e789bc4fa83f3389a20, user_id=b04ad85928b24b969e923ca20b2a453a, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.121801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ead484-fc60-413b-87dd-1ccb764259fb admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.128609 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb69bf8-2190-45cc-8d3e-d3d86c7a6fb5 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.136117 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750848c-92c1-43c1-9221-4151967cf85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.136749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750848c-92c1-43c1-9221-4151967cf85c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.137189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750848c-92c1-43c1-9221-4151967cf85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.137489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750848c-92c1-43c1-9221-4151967cf85c admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.138184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750848c-92c1-43c1-9221-4151967cf85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.140084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750848c-92c1-43c1-9221-4151967cf85c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.160922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 632/1267] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:16.161255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 636/1268] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:16 2022] PUT /identity/v3/projects/15fafabc29464e789bc4fa83f3389a20/users/b04ad85928b24b969e923ca20b2a453a/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:16.161358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750848c-92c1-43c1-9221-4151967cf85c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.161450 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ead484-fc60-413b-87dd-1ccb764259fb admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.184894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750848c-92c1-43c1-9221-4151967cf85c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.239050 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:16.251537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:16.266463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ead484-fc60-413b-87dd-1ccb764259fb admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.272777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d36a-91e4-4672-9d7e-37963ed09e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.273023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b8355-6dca-410f-a300-81843201b71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.273461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b8355-6dca-410f-a300-81843201b71f admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.273739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b8355-6dca-410f-a300-81843201b71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.274065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b8355-6dca-410f-a300-81843201b71f admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.274713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b8355-6dca-410f-a300-81843201b71f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.275138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d36a-91e4-4672-9d7e-37963ed09e7c admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.275610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d36a-91e4-4672-9d7e-37963ed09e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.276078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d36a-91e4-4672-9d7e-37963ed09e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.277153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80d36a-91e4-4672-9d7e-37963ed09e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.277401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b8355-6dca-410f-a300-81843201b71f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.279726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80d36a-91e4-4672-9d7e-37963ed09e7c admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.293063 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 637/1269] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:16.294318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d36a-91e4-4672-9d7e-37963ed09e7c admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.306516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 633/1270] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:16 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:16.309918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b8355-6dca-410f-a300-81843201b71f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.310137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d36a-91e4-4672-9d7e-37963ed09e7c admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.314978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cebc89-00f6-4d9e-972a-a4c98c13d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.315631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cebc89-00f6-4d9e-972a-a4c98c13d63a admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.316167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cebc89-00f6-4d9e-972a-a4c98c13d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.316492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cebc89-00f6-4d9e-972a-a4c98c13d63a admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.317217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cebc89-00f6-4d9e-972a-a4c98c13d63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.318856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cebc89-00f6-4d9e-972a-a4c98c13d63a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.323078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b8355-6dca-410f-a300-81843201b71f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.328905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53edf7-ac26-48a7-a48a-d7eb43e17ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.329695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 638/1271] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:16.330170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53edf7-ac26-48a7-a48a-d7eb43e17ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.330656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cebc89-00f6-4d9e-972a-a4c98c13d63a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.330832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53edf7-ac26-48a7-a48a-d7eb43e17ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.331285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53edf7-ac26-48a7-a48a-d7eb43e17ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.332034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53edf7-ac26-48a7-a48a-d7eb43e17ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.333849 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53edf7-ac26-48a7-a48a-d7eb43e17ba8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.346741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cebc89-00f6-4d9e-972a-a4c98c13d63a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.348690 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 634/1272] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:16.350071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ba6c4-b18a-47d8-bc92-d16c61df8ff4 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.350424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ba6c4-b18a-47d8-bc92-d16c61df8ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.350763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ba6c4-b18a-47d8-bc92-d16c61df8ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.356035 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa1f38-22b9-48c4-b10f-11270051caa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.356741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa1f38-22b9-48c4-b10f-11270051caa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.357652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa1f38-22b9-48c4-b10f-11270051caa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.357882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa1f38-22b9-48c4-b10f-11270051caa5 admin admin] PATH_INFO: `/v3/projects/cd52252f24114465af9bdf6e6fe68b76/users/1f624c36a3574b61bd17d5d73af71916/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.378442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa1f38-22b9-48c4-b10f-11270051caa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd52252f24114465af9bdf6e6fe68b76, user_id=1f624c36a3574b61bd17d5d73af71916, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.382452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-f36ba6c4-b18a-47d8-bc92-d16c61df8ff4 None None] MFA Rules not processed for user `b04ad85928b24b969e923ca20b2a453a`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:16.383754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f36ba6c4-b18a-47d8-bc92-d16c61df8ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:16.386721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa1f38-22b9-48c4-b10f-11270051caa5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.412798 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 639/1273] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:16 2022] PUT /identity/v3/projects/cd52252f24114465af9bdf6e6fe68b76/users/1f624c36a3574b61bd17d5d73af71916/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:16.413935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa1f38-22b9-48c4-b10f-11270051caa5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.460514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 635/1274] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:16.460514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ba6c4-b18a-47d8-bc92-d16c61df8ff4 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.474123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ba6c4-b18a-47d8-bc92-d16c61df8ff4 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.499308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa1f38-22b9-48c4-b10f-11270051caa5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.510224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f517a-fddb-402a-be11-59772edd9f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.511042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f517a-fddb-402a-be11-59772edd9f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.511323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f517a-fddb-402a-be11-59772edd9f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.511605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f517a-fddb-402a-be11-59772edd9f13 admin admin] PATH_INFO: `/v3/projects/8d0104a11c7241ee99ee49f32611807f/users/c3a261fe763b4e8583929c496ecf342a/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.531332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f517a-fddb-402a-be11-59772edd9f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0104a11c7241ee99ee49f32611807f, user_id=c3a261fe763b4e8583929c496ecf342a, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.535164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f517a-fddb-402a-be11-59772edd9f13 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.551355 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0d0dc-3eee-4043-bc45-d6a376109b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.552043 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0d0dc-3eee-4043-bc45-d6a376109b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.552313 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0d0dc-3eee-4043-bc45-d6a376109b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.552767 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0d0dc-3eee-4043-bc45-d6a376109b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.559929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 640/1275] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:16 2022] PUT /identity/v3/projects/8d0104a11c7241ee99ee49f32611807f/users/c3a261fe763b4e8583929c496ecf342a/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:16.560468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0d0dc-3eee-4043-bc45-d6a376109b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.560581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f517a-fddb-402a-be11-59772edd9f13 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.560673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0d0dc-3eee-4043-bc45-d6a376109b14 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.572721 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 636/1276] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:16.573206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0d0dc-3eee-4043-bc45-d6a376109b14 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.584814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0d0dc-3eee-4043-bc45-d6a376109b14 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.689846 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:16.692606 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f517a-fddb-402a-be11-59772edd9f13 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.703488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451073c-777c-4563-9df8-37e21fe9b8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.704130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451073c-777c-4563-9df8-37e21fe9b8ba admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.704409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451073c-777c-4563-9df8-37e21fe9b8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.707815 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451073c-777c-4563-9df8-37e21fe9b8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.708171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451073c-777c-4563-9df8-37e21fe9b8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.709891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451073c-777c-4563-9df8-37e21fe9b8ba admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.712194 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc436dd-fe05-43b6-8f52-c0a2b1a6a966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.716060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc436dd-fe05-43b6-8f52-c0a2b1a6a966 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.716981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc436dd-fe05-43b6-8f52-c0a2b1a6a966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.717272 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc436dd-fe05-43b6-8f52-c0a2b1a6a966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.718359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc436dd-fe05-43b6-8f52-c0a2b1a6a966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.720710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc436dd-fe05-43b6-8f52-c0a2b1a6a966 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.737368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 641/1278] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:16.737544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 637/1278] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:16.738555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451073c-777c-4563-9df8-37e21fe9b8ba admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.738900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646228d8-8f2e-4709-9b7c-8c19744630b1 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.739218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646228d8-8f2e-4709-9b7c-8c19744630b1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.739502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646228d8-8f2e-4709-9b7c-8c19744630b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.749948 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451073c-777c-4563-9df8-37e21fe9b8ba admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.756508 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8d388-5ca0-4002-b68d-72fb25530c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.757532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8d388-5ca0-4002-b68d-72fb25530c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.758015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8d388-5ca0-4002-b68d-72fb25530c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.758488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8d388-5ca0-4002-b68d-72fb25530c9e admin admin] PATH_INFO: `/v3/projects/633afaabbbff47a4a99ebc997e67b01a/users/916699a212eb4d8aa945f299ac0d4c96/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.761345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-646228d8-8f2e-4709-9b7c-8c19744630b1 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:16.763663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-646228d8-8f2e-4709-9b7c-8c19744630b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:16.771934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 638/1279] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:16.772994 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646228d8-8f2e-4709-9b7c-8c19744630b1 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.785691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646228d8-8f2e-4709-9b7c-8c19744630b1 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.792715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c397760-5e8d-4104-9917-456f450bfa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.793541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c397760-5e8d-4104-9917-456f450bfa0e admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.793923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c397760-5e8d-4104-9917-456f450bfa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.794211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c397760-5e8d-4104-9917-456f450bfa0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.795742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c397760-5e8d-4104-9917-456f450bfa0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.798242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c397760-5e8d-4104-9917-456f450bfa0e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.800745 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8d388-5ca0-4002-b68d-72fb25530c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=633afaabbbff47a4a99ebc997e67b01a, user_id=916699a212eb4d8aa945f299ac0d4c96, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.805370 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8d388-5ca0-4002-b68d-72fb25530c9e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.810906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:16.826791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 642/1280] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:16 2022] PUT /identity/v3/projects/633afaabbbff47a4a99ebc997e67b01a/users/916699a212eb4d8aa945f299ac0d4c96/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:16.828985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5edd2b-6fa1-4d6c-b6aa-cd4c01c5bc11 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.828985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5edd2b-6fa1-4d6c-b6aa-cd4c01c5bc11 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.828985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5edd2b-6fa1-4d6c-b6aa-cd4c01c5bc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.855218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 639/1281] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:16 2022] POST /identity/v3/projects => generated 355 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:16.855218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c397760-5e8d-4104-9917-456f450bfa0e admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.871352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-aa5edd2b-6fa1-4d6c-b6aa-cd4c01c5bc11 None None] MFA Rules not processed for user `c3a261fe763b4e8583929c496ecf342a`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:16.873394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-aa5edd2b-6fa1-4d6c-b6aa-cd4c01c5bc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:16.936239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c397760-5e8d-4104-9917-456f450bfa0e admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:16.941928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44892e48-d5fe-483c-b9c3-e0a138a331fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:16.942818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44892e48-d5fe-483c-b9c3-e0a138a331fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:16.943307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44892e48-d5fe-483c-b9c3-e0a138a331fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:16.943676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44892e48-d5fe-483c-b9c3-e0a138a331fc admin admin] PATH_INFO: `/v3/projects/c4144984961a4ec78c81659ce0ca5e9e/users/d96911960ec94380b902ea1d47bfa394/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:16.972823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44892e48-d5fe-483c-b9c3-e0a138a331fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4144984961a4ec78c81659ce0ca5e9e, user_id=d96911960ec94380b902ea1d47bfa394, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:16.977208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 643/1282] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:16.978806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44892e48-d5fe-483c-b9c3-e0a138a331fc admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:16.979210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5edd2b-6fa1-4d6c-b6aa-cd4c01c5bc11 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:16.994228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5edd2b-6fa1-4d6c-b6aa-cd4c01c5bc11 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:17.011844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 640/1283] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:16 2022] PUT /identity/v3/projects/c4144984961a4ec78c81659ce0ca5e9e/users/d96911960ec94380b902ea1d47bfa394/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:17.014066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44892e48-d5fe-483c-b9c3-e0a138a331fc admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.081457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a0338-3535-4955-9695-48dcbff3c7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:17.082608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a0338-3535-4955-9695-48dcbff3c7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.083117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a0338-3535-4955-9695-48dcbff3c7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.083524 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a0338-3535-4955-9695-48dcbff3c7f7 admin admin] PATH_INFO: `/v3/projects/6c580418dbc24525b6631dcd0a5f7876/users/47bc074aee7d427bb0336fd19d31ef9d/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.106655 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a0338-3535-4955-9695-48dcbff3c7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c580418dbc24525b6631dcd0a5f7876, user_id=47bc074aee7d427bb0336fd19d31ef9d, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:17.110565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a0338-3535-4955-9695-48dcbff3c7f7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:17.119428 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44892e48-d5fe-483c-b9c3-e0a138a331fc admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:17.126444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858534e-8d68-4451-b9b4-3e2bdc417d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:17.127407 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858534e-8d68-4451-b9b4-3e2bdc417d00 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.127885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858534e-8d68-4451-b9b4-3e2bdc417d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.128305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858534e-8d68-4451-b9b4-3e2bdc417d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.129191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858534e-8d68-4451-b9b4-3e2bdc417d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:17.131098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858534e-8d68-4451-b9b4-3e2bdc417d00 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:17.137444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 644/1284] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:16 2022] PUT /identity/v3/projects/6c580418dbc24525b6631dcd0a5f7876/users/47bc074aee7d427bb0336fd19d31ef9d/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:17.139481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a0338-3535-4955-9695-48dcbff3c7f7 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.143125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6b3a0338-3535-4955-9695-48dcbff3c7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:17.145961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 641/1285] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:17.147467 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892bf9c-746d-4df8-986f-bc66ed512b2a None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.147932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892bf9c-746d-4df8-986f-bc66ed512b2a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.148344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892bf9c-746d-4df8-986f-bc66ed512b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.171148 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-a892bf9c-746d-4df8-986f-bc66ed512b2a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:17.172711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a892bf9c-746d-4df8-986f-bc66ed512b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:17.230628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:17.237212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a0338-3535-4955-9695-48dcbff3c7f7 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:17.242968 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e715edc-5ed2-4262-8b5b-622ac54bec98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:17.243839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e715edc-5ed2-4262-8b5b-622ac54bec98 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.244581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e715edc-5ed2-4262-8b5b-622ac54bec98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.244940 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e715edc-5ed2-4262-8b5b-622ac54bec98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.274165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:17.280154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 642/1286] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:17.281257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892bf9c-746d-4df8-986f-bc66ed512b2a None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.298004 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892bf9c-746d-4df8-986f-bc66ed512b2a None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:17.303947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308663e-b40e-4c2b-8791-b91da5a15b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:17.305473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308663e-b40e-4c2b-8791-b91da5a15b96 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.306032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308663e-b40e-4c2b-8791-b91da5a15b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.306414 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308663e-b40e-4c2b-8791-b91da5a15b96 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.307191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308663e-b40e-4c2b-8791-b91da5a15b96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:17.309641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308663e-b40e-4c2b-8791-b91da5a15b96 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:17.355112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e715edc-5ed2-4262-8b5b-622ac54bec98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:17.358294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e715edc-5ed2-4262-8b5b-622ac54bec98 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:17.360308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 643/1287] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:17 2022] POST /identity/v3/users => generated 284 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:17.361914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49327e-6abe-4f16-90a9-0537342f2fb5 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.362223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49327e-6abe-4f16-90a9-0537342f2fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.362585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49327e-6abe-4f16-90a9-0537342f2fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.373755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 645/1288] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:17.377293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78717338-56ec-4a75-844a-968ae5231bf9 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.377578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78717338-56ec-4a75-844a-968ae5231bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.377850 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78717338-56ec-4a75-844a-968ae5231bf9 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.379364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 646/1289] 10.222.0.17 () {54 vars in 961 bytes} [Tue Oct 18 11:40:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:40:17.380755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09019fa9-aea2-483a-9095-93d191ebf2a1 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.381064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09019fa9-aea2-483a-9095-93d191ebf2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.381334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09019fa9-aea2-483a-9095-93d191ebf2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.404797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-09019fa9-aea2-483a-9095-93d191ebf2a1 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:17.406176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-09019fa9-aea2-483a-9095-93d191ebf2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:17.406320 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-cb49327e-6abe-4f16-90a9-0537342f2fb5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:17.407857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cb49327e-6abe-4f16-90a9-0537342f2fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:17.416964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 647/1290] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:17.417902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09019fa9-aea2-483a-9095-93d191ebf2a1 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.421276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 644/1291] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:17.425073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49327e-6abe-4f16-90a9-0537342f2fb5 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.433188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09019fa9-aea2-483a-9095-93d191ebf2a1 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:17.437779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49327e-6abe-4f16-90a9-0537342f2fb5 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:17.439460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10b7a0-b968-4201-bb82-0690234e2dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:17.440712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10b7a0-b968-4201-bb82-0690234e2dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.441982 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10b7a0-b968-4201-bb82-0690234e2dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.442324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10b7a0-b968-4201-bb82-0690234e2dbd admin admin] PATH_INFO: `/v3/projects/9b20565451464b3f8e227b1804e9fb92/users/8fe6229212e4411fb2f617a7a49a570d/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.443736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cad513-b06b-4e19-8c34-ab8e3d1c3cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:17.444615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cad513-b06b-4e19-8c34-ab8e3d1c3cda service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.445117 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cad513-b06b-4e19-8c34-ab8e3d1c3cda service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.446220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cad513-b06b-4e19-8c34-ab8e3d1c3cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.458626 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cad513-b06b-4e19-8c34-ab8e3d1c3cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:17.460260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cad513-b06b-4e19-8c34-ab8e3d1c3cda service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:17.465234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10b7a0-b968-4201-bb82-0690234e2dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b20565451464b3f8e227b1804e9fb92, user_id=8fe6229212e4411fb2f617a7a49a570d, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:17.469321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10b7a0-b968-4201-bb82-0690234e2dbd admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:17.473474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 645/1292] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:17.474484 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cad513-b06b-4e19-8c34-ab8e3d1c3cda service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.485465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cad513-b06b-4e19-8c34-ab8e3d1c3cda service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:17.491118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 648/1293] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:17 2022] PUT /identity/v3/projects/9b20565451464b3f8e227b1804e9fb92/users/8fe6229212e4411fb2f617a7a49a570d/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:17.493943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d71a4-3b99-47fa-8012-edebfd01c953 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.494394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d71a4-3b99-47fa-8012-edebfd01c953 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.494782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d71a4-3b99-47fa-8012-edebfd01c953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.565109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b07d71a4-3b99-47fa-8012-edebfd01c953 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:17.566969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b07d71a4-3b99-47fa-8012-edebfd01c953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:17.578437 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b153a3f-f79f-4620-8b7f-83c19384f12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:17.579220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b153a3f-f79f-4620-8b7f-83c19384f12c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.579600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b153a3f-f79f-4620-8b7f-83c19384f12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.579973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b153a3f-f79f-4620-8b7f-83c19384f12c admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.580778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b153a3f-f79f-4620-8b7f-83c19384f12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:17.582704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b153a3f-f79f-4620-8b7f-83c19384f12c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:17.605943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 646/1294] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:17.607215 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b153a3f-f79f-4620-8b7f-83c19384f12c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.671574 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 649/1295] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Oct 18 11:40:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:17.672909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d71a4-3b99-47fa-8012-edebfd01c953 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.703070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b153a3f-f79f-4620-8b7f-83c19384f12c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:17.713194 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109c667-db81-4a9c-afa6-d7a6838c15c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:17.714220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109c667-db81-4a9c-afa6-d7a6838c15c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.714754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109c667-db81-4a9c-afa6-d7a6838c15c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.715166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109c667-db81-4a9c-afa6-d7a6838c15c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.731096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109c667-db81-4a9c-afa6-d7a6838c15c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:17.732877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109c667-db81-4a9c-afa6-d7a6838c15c2 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:17.745626 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 647/1296] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:17.747240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d950db-7df6-4fe6-8a2c-eb691013a721 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.748676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d950db-7df6-4fe6-8a2c-eb691013a721 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.749173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d950db-7df6-4fe6-8a2c-eb691013a721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.777846 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d71a4-3b99-47fa-8012-edebfd01c953 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:17.783028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7307721-fea7-404c-87a1-92d4c4d136ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:17.783759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7307721-fea7-404c-87a1-92d4c4d136ed service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.784090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7307721-fea7-404c-87a1-92d4c4d136ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.784413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-74d950db-7df6-4fe6-8a2c-eb691013a721 None None] MFA Rules not processed for user `8fe6229212e4411fb2f617a7a49a570d`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:17.784545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7307721-fea7-404c-87a1-92d4c4d136ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.786072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-74d950db-7df6-4fe6-8a2c-eb691013a721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:17.796812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7307721-fea7-404c-87a1-92d4c4d136ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:17.799601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7307721-fea7-404c-87a1-92d4c4d136ed service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:17.824202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 650/1297] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:17.827059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7307721-fea7-404c-87a1-92d4c4d136ed service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.842377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7307721-fea7-404c-87a1-92d4c4d136ed service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:17.844017 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:17.850375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15937fef-2585-4c91-ad4a-8a11257b143d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:17.852385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937fef-2585-4c91-ad4a-8a11257b143d admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:17.852791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937fef-2585-4c91-ad4a-8a11257b143d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:17.853202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937fef-2585-4c91-ad4a-8a11257b143d admin admin] PATH_INFO: `/v3/projects/ba9c292aafb94661bd46780013913954/users/a540ae80623344d3bf7ec6517357d9f5/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:17.860600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:17.958561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15937fef-2585-4c91-ad4a-8a11257b143d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9c292aafb94661bd46780013913954, user_id=a540ae80623344d3bf7ec6517357d9f5, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:17.958561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15937fef-2585-4c91-ad4a-8a11257b143d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:17.959073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 648/1298] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:17.959073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d950db-7df6-4fe6-8a2c-eb691013a721 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.959770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 651/1299] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:17 2022] PUT /identity/v3/projects/ba9c292aafb94661bd46780013913954/users/a540ae80623344d3bf7ec6517357d9f5/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:17.960989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15937fef-2585-4c91-ad4a-8a11257b143d admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:17.965574 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d950db-7df6-4fe6-8a2c-eb691013a721 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:18.081811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1856f-4cae-468f-94a7-cad3203c5bf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:18.082453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1856f-4cae-468f-94a7-cad3203c5bf6 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.082728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1856f-4cae-468f-94a7-cad3203c5bf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.083004 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1856f-4cae-468f-94a7-cad3203c5bf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.131766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15937fef-2585-4c91-ad4a-8a11257b143d admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:18.138142 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767054aa-0ef5-4ae9-92fc-66645f1879fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:18.139486 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767054aa-0ef5-4ae9-92fc-66645f1879fb admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.139991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767054aa-0ef5-4ae9-92fc-66645f1879fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.140332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767054aa-0ef5-4ae9-92fc-66645f1879fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.141117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767054aa-0ef5-4ae9-92fc-66645f1879fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:18.143244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767054aa-0ef5-4ae9-92fc-66645f1879fb admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:18.157576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1856f-4cae-468f-94a7-cad3203c5bf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:18.160496 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1856f-4cae-468f-94a7-cad3203c5bf6 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:18.163881 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 652/1300] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:18.166190 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d2c1-9f5b-41b2-a431-96c6fb97e75d None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.166597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d2c1-9f5b-41b2-a431-96c6fb97e75d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.166936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d2c1-9f5b-41b2-a431-96c6fb97e75d None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.170137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 653/1301] 10.222.0.17 () {54 vars in 961 bytes} [Tue Oct 18 11:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:40:18.171362 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514d2c1-9f5b-41b2-a431-96c6fb97e75d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:18.193392 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 649/1302] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:18.195213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04eadc1-03e9-4bbb-8be2-a564ba84b374 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.197970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04eadc1-03e9-4bbb-8be2-a564ba84b374 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.198132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514d2c1-9f5b-41b2-a431-96c6fb97e75d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:18.198613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04eadc1-03e9-4bbb-8be2-a564ba84b374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.206289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba443c4d-f05c-4b34-b208-10747e89ea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:18.207592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba443c4d-f05c-4b34-b208-10747e89ea66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.208079 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba443c4d-f05c-4b34-b208-10747e89ea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.208563 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba443c4d-f05c-4b34-b208-10747e89ea66 admin admin] PATH_INFO: `/v3/projects/ba9c292aafb94661bd46780013913954/users/a540ae80623344d3bf7ec6517357d9f5/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.211603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba443c4d-f05c-4b34-b208-10747e89ea66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9c292aafb94661bd46780013913954, user_id=a540ae80623344d3bf7ec6517357d9f5, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:18.214768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba443c4d-f05c-4b34-b208-10747e89ea66 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:18.245800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d04eadc1-03e9-4bbb-8be2-a564ba84b374 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:18.253174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d04eadc1-03e9-4bbb-8be2-a564ba84b374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:18.271282 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 654/1303] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:18 2022] PUT /identity/v3/projects/ba9c292aafb94661bd46780013913954/users/a540ae80623344d3bf7ec6517357d9f5/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:18.273926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8989d0-63bb-4d6e-92d0-564b840c5ebe None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.274761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8989d0-63bb-4d6e-92d0-564b840c5ebe None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.275314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8989d0-63bb-4d6e-92d0-564b840c5ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.323614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-6c8989d0-63bb-4d6e-92d0-564b840c5ebe None None] MFA Rules not processed for user `a540ae80623344d3bf7ec6517357d9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:18.325297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6c8989d0-63bb-4d6e-92d0-564b840c5ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:18.387309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:18.405299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 650/1304] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Oct 18 11:40:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:18.413777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04eadc1-03e9-4bbb-8be2-a564ba84b374 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:18.436770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04eadc1-03e9-4bbb-8be2-a564ba84b374 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:18.450509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4373ba1-444a-42bc-b191-e320a9767747 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:18.453311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373ba1-444a-42bc-b191-e320a9767747 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.455757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373ba1-444a-42bc-b191-e320a9767747 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.456184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373ba1-444a-42bc-b191-e320a9767747 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.541754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 655/1305] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:18.559177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8989d0-63bb-4d6e-92d0-564b840c5ebe None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:18.645552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4373ba1-444a-42bc-b191-e320a9767747 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:18.648239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4373ba1-444a-42bc-b191-e320a9767747 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:18.662771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 651/1306] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:40:18.664327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8d83a-c617-470f-b9a0-b1457dd2332d None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.664668 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8d83a-c617-470f-b9a0-b1457dd2332d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.665002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8989d0-63bb-4d6e-92d0-564b840c5ebe None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:18.665350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8d83a-c617-470f-b9a0-b1457dd2332d None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.667283 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 652/1307] 10.222.0.17 () {54 vars in 912 bytes} [Tue Oct 18 11:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:40:18.680254 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465ef29-65ce-427a-9c91-dd29a0140a8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:18.680936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465ef29-65ce-427a-9c91-dd29a0140a8e service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.682690 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358610-8aff-4c73-942f-9c1efac52aa2 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.683642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358610-8aff-4c73-942f-9c1efac52aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.684427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358610-8aff-4c73-942f-9c1efac52aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.686733 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465ef29-65ce-427a-9c91-dd29a0140a8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.687122 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465ef29-65ce-427a-9c91-dd29a0140a8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.704864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:18.713570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465ef29-65ce-427a-9c91-dd29a0140a8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:18.718749 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465ef29-65ce-427a-9c91-dd29a0140a8e service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:18.753567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 656/1308] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:40:18.784800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-26358610-8aff-4c73-942f-9c1efac52aa2 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:18.786716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-26358610-8aff-4c73-942f-9c1efac52aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:18.816274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 653/1309] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:40:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:18.832541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465ef29-65ce-427a-9c91-dd29a0140a8e service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:18.842342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465ef29-65ce-427a-9c91-dd29a0140a8e service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:18.858244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfec01d-36a4-47ce-98ad-acaebe5b045b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:18.858872 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfec01d-36a4-47ce-98ad-acaebe5b045b service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:18.859479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfec01d-36a4-47ce-98ad-acaebe5b045b service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:18.863999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfec01d-36a4-47ce-98ad-acaebe5b045b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:18.906153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfec01d-36a4-47ce-98ad-acaebe5b045b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:18.907672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfec01d-36a4-47ce-98ad-acaebe5b045b service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:18.975838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 657/1310] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:18.991881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26358610-8aff-4c73-942f-9c1efac52aa2 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:19.004112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26358610-8aff-4c73-942f-9c1efac52aa2 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:19.023388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27e854-7734-4e83-81de-a0935db3103e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:19.031195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27e854-7734-4e83-81de-a0935db3103e service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:19.031869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27e854-7734-4e83-81de-a0935db3103e service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:19.032375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27e854-7734-4e83-81de-a0935db3103e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:19.199454 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27e854-7734-4e83-81de-a0935db3103e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:19.201085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27e854-7734-4e83-81de-a0935db3103e service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:19.235898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 654/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:19.353342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40f86b-ae77-49b7-9a8f-fc2ec8599038 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:19.354349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40f86b-ae77-49b7-9a8f-fc2ec8599038 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:19.354797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40f86b-ae77-49b7-9a8f-fc2ec8599038 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:19.356477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 658/1312] 10.222.0.17 () {54 vars in 912 bytes} [Tue Oct 18 11:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:40:19.364587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a3679-2c4c-49be-b741-94f1b885b5f7 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:19.365576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a3679-2c4c-49be-b741-94f1b885b5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:19.365576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a3679-2c4c-49be-b741-94f1b885b5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:19.423454 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-114a3679-2c4c-49be-b741-94f1b885b5f7 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:19.427127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-114a3679-2c4c-49be-b741-94f1b885b5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:19.458099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 655/1313] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:40:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:19.671992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40f86b-ae77-49b7-9a8f-fc2ec8599038 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:19.693803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40f86b-ae77-49b7-9a8f-fc2ec8599038 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:19.703689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dea85e-4e40-4ee8-a352-799e8820dc31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:19.703931 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dea85e-4e40-4ee8-a352-799e8820dc31 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:19.704080 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dea85e-4e40-4ee8-a352-799e8820dc31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:19.704361 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dea85e-4e40-4ee8-a352-799e8820dc31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:19.734044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dea85e-4e40-4ee8-a352-799e8820dc31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:19.738581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dea85e-4e40-4ee8-a352-799e8820dc31 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:19.782169 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 659/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:20.325361 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a3679-2c4c-49be-b741-94f1b885b5f7 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:20.493080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a3679-2c4c-49be-b741-94f1b885b5f7 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:20.507841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:20.508644 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:20.509023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:20.509490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] PATH_INFO: `/v3/users/8fe6229212e4411fb2f617a7a49a570d` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:20.511643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fe6229212e4411fb2f617a7a49a570d)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:20.513804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: WARNING py.warnings [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:40:20.513804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: warnings.warn(msg) Oct 18 11:40:20.513804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: Oct 18 11:40:20.519104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:20.650526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:20.694742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fe6229212e4411fb2f617a7a49a570d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:20.734979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fe6229212e4411fb2f617a7a49a570d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:20.735808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fe6229212e4411fb2f617a7a49a570d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:20.767303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fe6229212e4411fb2f617a7a49a570d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:20.768090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-db6fd7bf-bfed-469c-a9f4-16287b6c47cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fe6229212e4411fb2f617a7a49a570d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:20.790650 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 656/1315] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:20 2022] DELETE /identity/v3/users/8fe6229212e4411fb2f617a7a49a570d => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:20.792562 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fddb2a-bdd7-4b6a-a166-a36c6161bcd5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:20.793520 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fddb2a-bdd7-4b6a-a166-a36c6161bcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:20.794044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fddb2a-bdd7-4b6a-a166-a36c6161bcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:20.831351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-77fddb2a-bdd7-4b6a-a166-a36c6161bcd5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:20.834709 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-77fddb2a-bdd7-4b6a-a166-a36c6161bcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:20.945800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151d640-4344-4972-a963-f90f944dfb85 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:20.946089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151d640-4344-4972-a963-f90f944dfb85 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:20.946565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151d640-4344-4972-a963-f90f944dfb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:20.990823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 660/1316] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:40:21.011792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-b151d640-4344-4972-a963-f90f944dfb85 None None] MFA Rules not processed for user `d96911960ec94380b902ea1d47bfa394`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:21.016983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b151d640-4344-4972-a963-f90f944dfb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:21.056272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fddb2a-bdd7-4b6a-a166-a36c6161bcd5 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:21.058683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-77fddb2a-bdd7-4b6a-a166-a36c6161bcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:21.187044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 657/1317] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:21.188285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151d640-4344-4972-a963-f90f944dfb85 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:21.191459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b151d640-4344-4972-a963-f90f944dfb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:21.197802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fddb2a-bdd7-4b6a-a166-a36c6161bcd5 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:21.206753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8298dd5-b8c3-4725-850e-1a6e3a240055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:21.207741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8298dd5-b8c3-4725-850e-1a6e3a240055 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:21.208253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8298dd5-b8c3-4725-850e-1a6e3a240055 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:21.208747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8298dd5-b8c3-4725-850e-1a6e3a240055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:21.218126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151d640-4344-4972-a963-f90f944dfb85 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:21.225539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8298dd5-b8c3-4725-850e-1a6e3a240055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:21.227365 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8298dd5-b8c3-4725-850e-1a6e3a240055 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:21.230877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f9010-5f93-44a0-8ea0-24733b558799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:21.231562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9010-5f93-44a0-8ea0-24733b558799 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:21.231934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9010-5f93-44a0-8ea0-24733b558799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:21.232214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9010-5f93-44a0-8ea0-24733b558799 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:21.232875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f9010-5f93-44a0-8ea0-24733b558799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:21.234821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f9010-5f93-44a0-8ea0-24733b558799 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:21.240400 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 661/1318] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:21.301777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 658/1319] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:21 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:21.303641 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8298dd5-b8c3-4725-850e-1a6e3a240055 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:21.315079 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8298dd5-b8c3-4725-850e-1a6e3a240055 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:21.459636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002a10f-aaa9-480b-a821-210aa31f1441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:21.460128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002a10f-aaa9-480b-a821-210aa31f1441 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:21.460434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002a10f-aaa9-480b-a821-210aa31f1441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:21.460722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002a10f-aaa9-480b-a821-210aa31f1441 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:21.461367 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002a10f-aaa9-480b-a821-210aa31f1441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:21.463333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002a10f-aaa9-480b-a821-210aa31f1441 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:21.546460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f9010-5f93-44a0-8ea0-24733b558799 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:21.559104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 662/1320] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:21 2022] POST /identity/v3/users => generated 308 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:21.566397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f9010-5f93-44a0-8ea0-24733b558799 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:21.586134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc17441-e165-47e9-9a1e-a47a02c5d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:21.587469 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc17441-e165-47e9-9a1e-a47a02c5d420 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:21.588903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc17441-e165-47e9-9a1e-a47a02c5d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:21.590412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc17441-e165-47e9-9a1e-a47a02c5d420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:21.594856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc17441-e165-47e9-9a1e-a47a02c5d420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:21.598848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc17441-e165-47e9-9a1e-a47a02c5d420 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:21.633721 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 659/1321] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:40:21.639073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002a10f-aaa9-480b-a821-210aa31f1441 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:21.645160 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:21.660343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002a10f-aaa9-480b-a821-210aa31f1441 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:21.678975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075af18-06e1-4cbe-8d8b-d7e91f056ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:21.681389 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075af18-06e1-4cbe-8d8b-d7e91f056ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:21.683939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075af18-06e1-4cbe-8d8b-d7e91f056ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:21.686422 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075af18-06e1-4cbe-8d8b-d7e91f056ccf admin admin] PATH_INFO: `/v3/projects/625198a0d2734fc49ca2a6712a6bea07/users/8c5dd927177443dba4eeb820c318b2f0/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:21.749637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075af18-06e1-4cbe-8d8b-d7e91f056ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=625198a0d2734fc49ca2a6712a6bea07, user_id=8c5dd927177443dba4eeb820c318b2f0, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:21.762082 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075af18-06e1-4cbe-8d8b-d7e91f056ccf admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:21.810262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 663/1322] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:21 2022] PUT /identity/v3/projects/625198a0d2734fc49ca2a6712a6bea07/users/8c5dd927177443dba4eeb820c318b2f0/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:21.815213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc17441-e165-47e9-9a1e-a47a02c5d420 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:21.833692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc17441-e165-47e9-9a1e-a47a02c5d420 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:21.987731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be6ded-2ff1-443d-a5d8-f1864bd2648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:21.995124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be6ded-2ff1-443d-a5d8-f1864bd2648a admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:21.995461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be6ded-2ff1-443d-a5d8-f1864bd2648a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:21.995741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be6ded-2ff1-443d-a5d8-f1864bd2648a admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:22.000821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be6ded-2ff1-443d-a5d8-f1864bd2648a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:22.006288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be6ded-2ff1-443d-a5d8-f1864bd2648a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:22.009201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:22.029591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 660/1323] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:22.030691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075af18-06e1-4cbe-8d8b-d7e91f056ccf admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:22.049453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075af18-06e1-4cbe-8d8b-d7e91f056ccf admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:22.063134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492144d4-15b3-4ebc-9807-6c0a6f324789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:22.066289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492144d4-15b3-4ebc-9807-6c0a6f324789 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:22.068012 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492144d4-15b3-4ebc-9807-6c0a6f324789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:22.069222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492144d4-15b3-4ebc-9807-6c0a6f324789 admin admin] PATH_INFO: `/v3/projects/625198a0d2734fc49ca2a6712a6bea07/users/8c5dd927177443dba4eeb820c318b2f0/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:22.076827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492144d4-15b3-4ebc-9807-6c0a6f324789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625198a0d2734fc49ca2a6712a6bea07, user_id=8c5dd927177443dba4eeb820c318b2f0, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:22.087512 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492144d4-15b3-4ebc-9807-6c0a6f324789 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:22.142250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 664/1324] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:22 2022] PUT /identity/v3/projects/625198a0d2734fc49ca2a6712a6bea07/users/8c5dd927177443dba4eeb820c318b2f0/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:23.373226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be6ded-2ff1-443d-a5d8-f1864bd2648a admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:23.375554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c6be6ded-2ff1-443d-a5d8-f1864bd2648a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:23.395110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be6ded-2ff1-443d-a5d8-f1864bd2648a admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:23.551495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:23.552159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:23.552439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:23.554451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] PATH_INFO: `/v3/projects/9b20565451464b3f8e227b1804e9fb92` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:23.557475 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b20565451464b3f8e227b1804e9fb92)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:23.562738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:23.760017 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b20565451464b3f8e227b1804e9fb92', 'request_id': 'req-7bf4056b-009d-4461-a584-acc7b4875566'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:23.769685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b20565451464b3f8e227b1804e9fb92', 'request_id': 'req-7bf4056b-009d-4461-a584-acc7b4875566'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:23.784134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b20565451464b3f8e227b1804e9fb92', 'request_id': 'req-7bf4056b-009d-4461-a584-acc7b4875566'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:23.896565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] The token cache is being invalidate because project 9b20565451464b3f8e227b1804e9fb92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:23.897960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:23.916190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7bf4056b-009d-4461-a584-acc7b4875566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:23.938573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80b58-5530-4296-9b19-b2c2c933c0da None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:23.939560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80b58-5530-4296-9b19-b2c2c933c0da None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:23.940648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d80b58-5530-4296-9b19-b2c2c933c0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:23.945114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 661/1325] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:23 2022] DELETE /identity/v3/projects/9b20565451464b3f8e227b1804e9fb92 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:23.976410 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-86d80b58-5530-4296-9b19-b2c2c933c0da None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:23.979747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-86d80b58-5530-4296-9b19-b2c2c933c0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:24.144488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 665/1326] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:24.152368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569c88f-1d1d-444f-9d81-8d2b7fcd0646 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:24.152655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569c88f-1d1d-444f-9d81-8d2b7fcd0646 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:24.152960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569c88f-1d1d-444f-9d81-8d2b7fcd0646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:24.187392 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c569c88f-1d1d-444f-9d81-8d2b7fcd0646 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:24.188639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c569c88f-1d1d-444f-9d81-8d2b7fcd0646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:24.220382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d80b58-5530-4296-9b19-b2c2c933c0da None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:24.229512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 662/1327] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:24.237815 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d80b58-5530-4296-9b19-b2c2c933c0da None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:24.270898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05fee9-c2a4-4a77-925f-1a52fb5ef089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:24.272720 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05fee9-c2a4-4a77-925f-1a52fb5ef089 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:24.273426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05fee9-c2a4-4a77-925f-1a52fb5ef089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:24.273426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05fee9-c2a4-4a77-925f-1a52fb5ef089 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:24.273979 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05fee9-c2a4-4a77-925f-1a52fb5ef089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:24.274288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05fee9-c2a4-4a77-925f-1a52fb5ef089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:24.278834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05fee9-c2a4-4a77-925f-1a52fb5ef089 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:24.326352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569c88f-1d1d-444f-9d81-8d2b7fcd0646 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:24.341240 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 666/1328] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:24.342042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569c88f-1d1d-444f-9d81-8d2b7fcd0646 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:24.348329 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a73835-779b-4f19-a5b9-35c598820318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:24.349813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a73835-779b-4f19-a5b9-35c598820318 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:24.350105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a73835-779b-4f19-a5b9-35c598820318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:24.350379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a73835-779b-4f19-a5b9-35c598820318 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:24.351027 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a73835-779b-4f19-a5b9-35c598820318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:24.354301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a73835-779b-4f19-a5b9-35c598820318 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:24.406108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 663/1329] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:24 2022] POST /identity/v3/projects => generated 365 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:24.409011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05fee9-c2a4-4a77-925f-1a52fb5ef089 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:24.423525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05fee9-c2a4-4a77-925f-1a52fb5ef089 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:24.499266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e0f56-9b67-4fff-b5dc-49533ea01239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:24.501760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e0f56-9b67-4fff-b5dc-49533ea01239 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:24.501760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e0f56-9b67-4fff-b5dc-49533ea01239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:24.501760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e0f56-9b67-4fff-b5dc-49533ea01239 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:24.501760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e0f56-9b67-4fff-b5dc-49533ea01239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:24.503595 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e0f56-9b67-4fff-b5dc-49533ea01239 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:24.560614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 667/1330] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:24 2022] POST /identity/v3/users => generated 283 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:24.564420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a73835-779b-4f19-a5b9-35c598820318 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:24.582435 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a73835-779b-4f19-a5b9-35c598820318 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:24.597255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51127d9d-8d95-4709-b277-abc75aa9fe3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:24.597894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51127d9d-8d95-4709-b277-abc75aa9fe3d admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:24.598327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51127d9d-8d95-4709-b277-abc75aa9fe3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:24.598599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51127d9d-8d95-4709-b277-abc75aa9fe3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:24.599430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51127d9d-8d95-4709-b277-abc75aa9fe3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:24.601261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51127d9d-8d95-4709-b277-abc75aa9fe3d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:24.632809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 664/1331] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:24.634321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e0f56-9b67-4fff-b5dc-49533ea01239 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:24.663105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e0f56-9b67-4fff-b5dc-49533ea01239 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:24.677537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65657b-63ae-48b6-960b-099653116812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:24.678687 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65657b-63ae-48b6-960b-099653116812 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:24.679078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65657b-63ae-48b6-960b-099653116812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:24.681622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65657b-63ae-48b6-960b-099653116812 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:24.690737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65657b-63ae-48b6-960b-099653116812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:24.697523 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65657b-63ae-48b6-960b-099653116812 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:24.735612 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 668/1332] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:24.737010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51127d9d-8d95-4709-b277-abc75aa9fe3d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:24.753410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51127d9d-8d95-4709-b277-abc75aa9fe3d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:24.772228 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acec054-c8e3-49b4-88e9-6714abc491ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:24.773277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acec054-c8e3-49b4-88e9-6714abc491ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:24.773730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acec054-c8e3-49b4-88e9-6714abc491ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:24.774348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acec054-c8e3-49b4-88e9-6714abc491ca admin admin] PATH_INFO: `/v3/projects/b6e769716dc4464bb37102f6f6dcd033/users/3827950796f74cbf9aa29c6993da2b85/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:24.805316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acec054-c8e3-49b4-88e9-6714abc491ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e769716dc4464bb37102f6f6dcd033, user_id=3827950796f74cbf9aa29c6993da2b85, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:24.808736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acec054-c8e3-49b4-88e9-6714abc491ca admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:24.843464 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 665/1333] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:24 2022] PUT /identity/v3/projects/b6e769716dc4464bb37102f6f6dcd033/users/3827950796f74cbf9aa29c6993da2b85/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:24.856037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978a8ce-3415-49b4-a6dc-e8c0e657a9ae None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:24.856417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978a8ce-3415-49b4-a6dc-e8c0e657a9ae None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:24.857490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978a8ce-3415-49b4-a6dc-e8c0e657a9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:24.916067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a978a8ce-3415-49b4-a6dc-e8c0e657a9ae None None] MFA Rules not processed for user `3827950796f74cbf9aa29c6993da2b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:24.921511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a978a8ce-3415-49b4-a6dc-e8c0e657a9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:24.951919 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:25.057518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acec054-c8e3-49b4-88e9-6714abc491ca admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:25.074797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 669/1334] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:25.100557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:25.206930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acec054-c8e3-49b4-88e9-6714abc491ca admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:25.242764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf08b18-4855-4fa9-9bb6-75e89a0cbf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:25.243398 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf08b18-4855-4fa9-9bb6-75e89a0cbf3c admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:25.243679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf08b18-4855-4fa9-9bb6-75e89a0cbf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:25.244701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf08b18-4855-4fa9-9bb6-75e89a0cbf3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:25.245559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf08b18-4855-4fa9-9bb6-75e89a0cbf3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:25.250035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf08b18-4855-4fa9-9bb6-75e89a0cbf3c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:25.353140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb763a-a51c-4bd7-bf50-5088727ac08b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:25.353467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb763a-a51c-4bd7-bf50-5088727ac08b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:25.353738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb763a-a51c-4bd7-bf50-5088727ac08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:25.367461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 666/1335] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:25 2022] POST /identity/v3/projects => generated 365 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:25.372138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf08b18-4855-4fa9-9bb6-75e89a0cbf3c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:25.425806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-12cb763a-a51c-4bd7-bf50-5088727ac08b None None] MFA Rules not processed for user `8c5dd927177443dba4eeb820c318b2f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:25.427102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-12cb763a-a51c-4bd7-bf50-5088727ac08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:25.520311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf08b18-4855-4fa9-9bb6-75e89a0cbf3c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:25.534883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc3999-382c-414a-af60-9d95fa47885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:25.535580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc3999-382c-414a-af60-9d95fa47885b admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:25.535864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc3999-382c-414a-af60-9d95fa47885b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:25.536951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc3999-382c-414a-af60-9d95fa47885b admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:25.536951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc3999-382c-414a-af60-9d95fa47885b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:25.538851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc3999-382c-414a-af60-9d95fa47885b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:25.583936 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 667/1336] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:25 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:25.589392 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc3999-382c-414a-af60-9d95fa47885b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:25.606024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc3999-382c-414a-af60-9d95fa47885b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:25.612371 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04522a7c-717e-49f7-90c6-a9caff9ae8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:25.613231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04522a7c-717e-49f7-90c6-a9caff9ae8ca admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:25.613814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04522a7c-717e-49f7-90c6-a9caff9ae8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:25.614094 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04522a7c-717e-49f7-90c6-a9caff9ae8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:25.614928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04522a7c-717e-49f7-90c6-a9caff9ae8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:25.620818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04522a7c-717e-49f7-90c6-a9caff9ae8ca admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:25.631701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 670/1337] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:25 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:25.658832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 668/1338] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:25.662891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb763a-a51c-4bd7-bf50-5088727ac08b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:25.678013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb763a-a51c-4bd7-bf50-5088727ac08b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:25.708614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b86f-c7b6-41d6-ba34-9acb435db15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:25.709353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b86f-c7b6-41d6-ba34-9acb435db15d admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:25.709636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b86f-c7b6-41d6-ba34-9acb435db15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:25.710072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b86f-c7b6-41d6-ba34-9acb435db15d admin admin] PATH_INFO: `/v3/projects/85a6a15728fa45058b592d7802531219/users/396ca9ae07024ceb99e877995910e4cb/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:25.761348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44b86f-c7b6-41d6-ba34-9acb435db15d admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a6a15728fa45058b592d7802531219, user_id=396ca9ae07024ceb99e877995910e4cb, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:25.766364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44b86f-c7b6-41d6-ba34-9acb435db15d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:25.791637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 671/1339] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:25 2022] PUT /identity/v3/projects/85a6a15728fa45058b592d7802531219/users/396ca9ae07024ceb99e877995910e4cb/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:25.793158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04522a7c-717e-49f7-90c6-a9caff9ae8ca admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:25.807146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04522a7c-717e-49f7-90c6-a9caff9ae8ca admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:25.917119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5192df-5bc9-4b45-9b29-8ce82146cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:25.917980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5192df-5bc9-4b45-9b29-8ce82146cc03 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:25.918682 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5192df-5bc9-4b45-9b29-8ce82146cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:25.918922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5192df-5bc9-4b45-9b29-8ce82146cc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:25.920197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5192df-5bc9-4b45-9b29-8ce82146cc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:25.921914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5192df-5bc9-4b45-9b29-8ce82146cc03 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:25.942080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 669/1340] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:25.946222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b86f-c7b6-41d6-ba34-9acb435db15d admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:25.958701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b86f-c7b6-41d6-ba34-9acb435db15d admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:25.964607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a007d0-1f9c-43a7-a238-4e8831814f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:25.965324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a007d0-1f9c-43a7-a238-4e8831814f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:25.965900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a007d0-1f9c-43a7-a238-4e8831814f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:25.966827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a007d0-1f9c-43a7-a238-4e8831814f8f admin admin] PATH_INFO: `/v3/projects/85a6a15728fa45058b592d7802531219/users/396ca9ae07024ceb99e877995910e4cb/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:25.969984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a007d0-1f9c-43a7-a238-4e8831814f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a6a15728fa45058b592d7802531219, user_id=396ca9ae07024ceb99e877995910e4cb, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:25.973732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a007d0-1f9c-43a7-a238-4e8831814f8f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:26.025193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 672/1341] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:25 2022] PUT /identity/v3/projects/85a6a15728fa45058b592d7802531219/users/396ca9ae07024ceb99e877995910e4cb/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:26.026927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f4aa3-b151-4c60-b9f7-83eb46f1e61e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:26.027443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f4aa3-b151-4c60-b9f7-83eb46f1e61e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:26.028296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f4aa3-b151-4c60-b9f7-83eb46f1e61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:26.126619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-fd5f4aa3-b151-4c60-b9f7-83eb46f1e61e None None] MFA Rules not processed for user `396ca9ae07024ceb99e877995910e4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:26.128716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-fd5f4aa3-b151-4c60-b9f7-83eb46f1e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:26.145410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:26.334393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 670/1342] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:40:26.361350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a007d0-1f9c-43a7-a238-4e8831814f8f admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:26.365325 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-03a007d0-1f9c-43a7-a238-4e8831814f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:26.515328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a007d0-1f9c-43a7-a238-4e8831814f8f admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:26.533126 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3345df-e9c5-4252-a430-289d16a65191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:26.534496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3345df-e9c5-4252-a430-289d16a65191 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:26.535893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3345df-e9c5-4252-a430-289d16a65191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:26.536540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3345df-e9c5-4252-a430-289d16a65191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:26.569005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3345df-e9c5-4252-a430-289d16a65191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:26.570983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3345df-e9c5-4252-a430-289d16a65191 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:26.616347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 673/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:27.075794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01e2f-b26e-43a2-8638-5e61791c9e5b None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:27.076746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01e2f-b26e-43a2-8638-5e61791c9e5b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:27.077201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01e2f-b26e-43a2-8638-5e61791c9e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:27.192260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-0cd01e2f-b26e-43a2-8638-5e61791c9e5b None None] MFA Rules not processed for user `bddd3d2bb8f647f2ba616141eb16622e`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:27.193846 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0cd01e2f-b26e-43a2-8638-5e61791c9e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:27.430970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 671/1344] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:27.449940 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3345df-e9c5-4252-a430-289d16a65191 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:27.451907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5c3345df-e9c5-4252-a430-289d16a65191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:27.516319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:27.779104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3345df-e9c5-4252-a430-289d16a65191 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:27.803345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1b1ed-95b0-484e-96d6-1a709fc314ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:27.803909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1b1ed-95b0-484e-96d6-1a709fc314ba service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:27.804188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1b1ed-95b0-484e-96d6-1a709fc314ba service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:27.804453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1b1ed-95b0-484e-96d6-1a709fc314ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:27.840182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1b1ed-95b0-484e-96d6-1a709fc314ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:27.844622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1b1ed-95b0-484e-96d6-1a709fc314ba service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:27.880269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 674/1345] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:27.958474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01e2f-b26e-43a2-8638-5e61791c9e5b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:28.149554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01e2f-b26e-43a2-8638-5e61791c9e5b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:28.162063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:28.163034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:28.163465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:28.163908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] PATH_INFO: `/v3/users/3827950796f74cbf9aa29c6993da2b85` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:28.171350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3827950796f74cbf9aa29c6993da2b85)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:28.173770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:28.265655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:28.286754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3827950796f74cbf9aa29c6993da2b85'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:28.306735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3827950796f74cbf9aa29c6993da2b85'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:28.307783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3827950796f74cbf9aa29c6993da2b85'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:28.318793 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3827950796f74cbf9aa29c6993da2b85'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:28.319841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3827950796f74cbf9aa29c6993da2b85'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:28.357814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 672/1346] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:27 2022] DELETE /identity/v3/users/3827950796f74cbf9aa29c6993da2b85 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:28.359588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df474a70-7b32-4798-9109-c992b539c303 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:28.359878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df474a70-7b32-4798-9109-c992b539c303 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:28.360152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df474a70-7b32-4798-9109-c992b539c303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:28.405702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-df474a70-7b32-4798-9109-c992b539c303 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:28.407087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-df474a70-7b32-4798-9109-c992b539c303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:28.580155 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 675/1347] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:28.619922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:28.621445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:28.814809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2467e8-3d0a-4850-aeb3-f73b3e1f8271 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:28.836392 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a24457-677b-4728-8111-3e4245b59ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:28.842982 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a24457-677b-4728-8111-3e4245b59ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:28.842982 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a24457-677b-4728-8111-3e4245b59ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:28.842982 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a24457-677b-4728-8111-3e4245b59ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:28.885784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a24457-677b-4728-8111-3e4245b59ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:28.887263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a24457-677b-4728-8111-3e4245b59ba2 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:28.927512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 673/1348] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:28.941091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df474a70-7b32-4798-9109-c992b539c303 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:28.943833 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-df474a70-7b32-4798-9109-c992b539c303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:28.974638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df474a70-7b32-4798-9109-c992b539c303 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:28.998658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:28.999312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:28.999585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:28.999852 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] PATH_INFO: `/v3/users/c3a261fe763b4e8583929c496ecf342a` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:29.005812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a261fe763b4e8583929c496ecf342a)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:29.008044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:29.128642 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a261fe763b4e8583929c496ecf342a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:29.153233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a261fe763b4e8583929c496ecf342a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:29.155826 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a261fe763b4e8583929c496ecf342a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:29.197122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80b22e-7e12-4764-8722-51b1166074a3 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:29.197544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80b22e-7e12-4764-8722-51b1166074a3 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:29.197777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80b22e-7e12-4764-8722-51b1166074a3 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:29.199698 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 674/1349] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:40:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:40:29.205021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a261fe763b4e8583929c496ecf342a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:29.223390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8def2198-09f7-4dc4-8fd7-ee2c40ece68f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a261fe763b4e8583929c496ecf342a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:29.226258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ab596-4e83-41c1-ace4-49b953706f9a None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:29.226544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ab596-4e83-41c1-ace4-49b953706f9a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:29.227036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ab596-4e83-41c1-ace4-49b953706f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:29.235812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 676/1350] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:28 2022] DELETE /identity/v3/users/c3a261fe763b4e8583929c496ecf342a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:29.241161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc79c0-9b54-40b7-9833-932c87a3f055 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:29.243888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc79c0-9b54-40b7-9833-932c87a3f055 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:29.244270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc79c0-9b54-40b7-9833-932c87a3f055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:29.279510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-41fc79c0-9b54-40b7-9833-932c87a3f055 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:29.281571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-41fc79c0-9b54-40b7-9833-932c87a3f055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:29.284901 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-cc3ab596-4e83-41c1-ace4-49b953706f9a None None] MFA Rules not processed for user `99ec4a4276f345c2998723e554ab9608`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:29.286172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cc3ab596-4e83-41c1-ace4-49b953706f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:29.288541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:29.419055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 677/1351] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:29.435518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 675/1352] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:40:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:40:29.469412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc79c0-9b54-40b7-9833-932c87a3f055 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:29.471625 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-41fc79c0-9b54-40b7-9833-932c87a3f055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:29.477594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ab596-4e83-41c1-ace4-49b953706f9a None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:29.479034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cc3ab596-4e83-41c1-ace4-49b953706f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:29.497661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ab596-4e83-41c1-ace4-49b953706f9a None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:29.515138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb030c61-65ae-48ea-8b29-43e0989a3997 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:29.515774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb030c61-65ae-48ea-8b29-43e0989a3997 service placement] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:29.516057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb030c61-65ae-48ea-8b29-43e0989a3997 service placement] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:29.516328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb030c61-65ae-48ea-8b29-43e0989a3997 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:29.543702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb030c61-65ae-48ea-8b29-43e0989a3997 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:29.545379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb030c61-65ae-48ea-8b29-43e0989a3997 service placement] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:29.566432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 676/1353] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:40:29.581952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc79c0-9b54-40b7-9833-932c87a3f055 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:29.587073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042d985-dc82-4028-81a5-9651a3d1b81a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:29.587712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042d985-dc82-4028-81a5-9651a3d1b81a service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:29.587956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042d985-dc82-4028-81a5-9651a3d1b81a service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:29.588224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042d985-dc82-4028-81a5-9651a3d1b81a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:29.611650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042d985-dc82-4028-81a5-9651a3d1b81a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:29.612669 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042d985-dc82-4028-81a5-9651a3d1b81a service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:29.639739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 678/1354] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:30.929425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb030c61-65ae-48ea-8b29-43e0989a3997 service placement] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:30.930961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-eb030c61-65ae-48ea-8b29-43e0989a3997 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:30.960456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb030c61-65ae-48ea-8b29-43e0989a3997 service placement] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:30.985337 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:30.986239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:30.986594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:30.986879 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] PATH_INFO: `/v3/projects/8d0104a11c7241ee99ee49f32611807f` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:31.000114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0104a11c7241ee99ee49f32611807f)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:31.011103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:31.139648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:31.228644 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0104a11c7241ee99ee49f32611807f', 'request_id': 'req-2aeb96d1-2cc0-440b-9b17-a243962bd277'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:31.284441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0104a11c7241ee99ee49f32611807f', 'request_id': 'req-2aeb96d1-2cc0-440b-9b17-a243962bd277'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:31.323201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0104a11c7241ee99ee49f32611807f', 'request_id': 'req-2aeb96d1-2cc0-440b-9b17-a243962bd277'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:31.347330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] The token cache is being invalidate because project 8d0104a11c7241ee99ee49f32611807f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:31.347637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:31.348388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2aeb96d1-2cc0-440b-9b17-a243962bd277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:31.354120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 677/1355] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:30 2022] DELETE /identity/v3/projects/8d0104a11c7241ee99ee49f32611807f => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:31.356857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4952033-b014-40bd-8a1b-16e8f8b4b109 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:31.357339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4952033-b014-40bd-8a1b-16e8f8b4b109 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:31.357435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4952033-b014-40bd-8a1b-16e8f8b4b109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:31.416189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d4952033-b014-40bd-8a1b-16e8f8b4b109 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:31.427081 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d4952033-b014-40bd-8a1b-16e8f8b4b109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:31.632031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 679/1356] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:31.640600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c986caf-f3bf-467c-b4d5-610ffd7b5ba7 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:31.640966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c986caf-f3bf-467c-b4d5-610ffd7b5ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:31.641187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c986caf-f3bf-467c-b4d5-610ffd7b5ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:31.675022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-4c986caf-f3bf-467c-b4d5-610ffd7b5ba7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:31.676804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4c986caf-f3bf-467c-b4d5-610ffd7b5ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:31.700997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 678/1357] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:31.702943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4952033-b014-40bd-8a1b-16e8f8b4b109 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:31.714920 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4952033-b014-40bd-8a1b-16e8f8b4b109 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:31.734704 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e14344-b423-4d92-99c3-75b954c1a8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:31.735599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e14344-b423-4d92-99c3-75b954c1a8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:31.736033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e14344-b423-4d92-99c3-75b954c1a8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:31.736455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e14344-b423-4d92-99c3-75b954c1a8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:31.737330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e14344-b423-4d92-99c3-75b954c1a8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:31.737800 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e14344-b423-4d92-99c3-75b954c1a8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:31.739569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e14344-b423-4d92-99c3-75b954c1a8d4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:31.776385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 680/1358] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:31.784016 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c986caf-f3bf-467c-b4d5-610ffd7b5ba7 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:31.797444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c986caf-f3bf-467c-b4d5-610ffd7b5ba7 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:31.807687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3d9d6-a7d3-4506-865e-a3552ed9b5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:31.813456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3d9d6-a7d3-4506-865e-a3552ed9b5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:31.813748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3d9d6-a7d3-4506-865e-a3552ed9b5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:31.814026 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3d9d6-a7d3-4506-865e-a3552ed9b5d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:31.814705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3d9d6-a7d3-4506-865e-a3552ed9b5d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:31.816653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3d9d6-a7d3-4506-865e-a3552ed9b5d2 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:31.837290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7543fd9-800d-4f86-8c88-ce750016fa24 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:31.839311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7543fd9-800d-4f86-8c88-ce750016fa24 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:31.839891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7543fd9-800d-4f86-8c88-ce750016fa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:31.890770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 679/1359] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:31 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:31.894878 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3d9d6-a7d3-4506-865e-a3552ed9b5d2 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:31.905483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b7543fd9-800d-4f86-8c88-ce750016fa24 None None] MFA Rules not processed for user `47bc074aee7d427bb0336fd19d31ef9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:31.906948 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b7543fd9-800d-4f86-8c88-ce750016fa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:31.957072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:32.026753 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3d9d6-a7d3-4506-865e-a3552ed9b5d2 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:32.038463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c367807-aff9-4031-9405-179cf2545134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:32.039449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c367807-aff9-4031-9405-179cf2545134 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:32.040019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c367807-aff9-4031-9405-179cf2545134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:32.040492 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c367807-aff9-4031-9405-179cf2545134 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:32.041547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c367807-aff9-4031-9405-179cf2545134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:32.043778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c367807-aff9-4031-9405-179cf2545134 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:32.048622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 681/1360] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:32.062057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24fd37-10e8-41fb-9c31-cae7a2321008 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:32.062057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24fd37-10e8-41fb-9c31-cae7a2321008 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:32.062362 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24fd37-10e8-41fb-9c31-cae7a2321008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:32.087976 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 680/1361] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:31 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:32.092550 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c367807-aff9-4031-9405-179cf2545134 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:32.107518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-8c24fd37-10e8-41fb-9c31-cae7a2321008 None None] MFA Rules not processed for user `47bc074aee7d427bb0336fd19d31ef9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:32.108755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8c24fd37-10e8-41fb-9c31-cae7a2321008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:32.117276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c367807-aff9-4031-9405-179cf2545134 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:32.118884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 682/1362] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:40:32.133307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24fd37-10e8-41fb-9c31-cae7a2321008 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:32.134643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679c708-b441-4a61-a93a-fda46d008f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:32.135557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679c708-b441-4a61-a93a-fda46d008f33 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:32.135940 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679c708-b441-4a61-a93a-fda46d008f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:32.136334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679c708-b441-4a61-a93a-fda46d008f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:32.137667 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679c708-b441-4a61-a93a-fda46d008f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:32.140150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8c24fd37-10e8-41fb-9c31-cae7a2321008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:32.143903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679c708-b441-4a61-a93a-fda46d008f33 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:32.196434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 681/1363] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:32.197659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679c708-b441-4a61-a93a-fda46d008f33 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:32.228920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679c708-b441-4a61-a93a-fda46d008f33 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:32.236289 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16769b-98ee-4008-8ab9-ba906f71a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:32.237005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16769b-98ee-4008-8ab9-ba906f71a543 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:32.237332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16769b-98ee-4008-8ab9-ba906f71a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:32.237613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16769b-98ee-4008-8ab9-ba906f71a543 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:32.239098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16769b-98ee-4008-8ab9-ba906f71a543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:32.240989 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16769b-98ee-4008-8ab9-ba906f71a543 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:32.268887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 682/1364] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:32.270208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16769b-98ee-4008-8ab9-ba906f71a543 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:32.327519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16769b-98ee-4008-8ab9-ba906f71a543 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:32.361032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd5257-2e70-4b0d-83ec-61adb8fa3c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:32.368238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd5257-2e70-4b0d-83ec-61adb8fa3c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:32.368469 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24fd37-10e8-41fb-9c31-cae7a2321008 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:32.369055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd5257-2e70-4b0d-83ec-61adb8fa3c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:32.369055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd5257-2e70-4b0d-83ec-61adb8fa3c0c admin admin] PATH_INFO: `/v3/projects/a49aaaef7c9140539fb2704baf87c159/users/da571ebd580349c1bf9af12bf8ec4fdf/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:32.377685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0a9dd-0d1c-4e83-bc25-c24757f8e91c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:32.378337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0a9dd-0d1c-4e83-bc25-c24757f8e91c service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:32.378619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0a9dd-0d1c-4e83-bc25-c24757f8e91c service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:32.378893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0a9dd-0d1c-4e83-bc25-c24757f8e91c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:32.395490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0a9dd-0d1c-4e83-bc25-c24757f8e91c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:32.397153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0a9dd-0d1c-4e83-bc25-c24757f8e91c service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:32.414906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 683/1365] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:32.445773 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd5257-2e70-4b0d-83ec-61adb8fa3c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49aaaef7c9140539fb2704baf87c159, user_id=da571ebd580349c1bf9af12bf8ec4fdf, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:32.448718 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd5257-2e70-4b0d-83ec-61adb8fa3c0c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:32.483396 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 683/1366] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:32 2022] PUT /identity/v3/projects/a49aaaef7c9140539fb2704baf87c159/users/da571ebd580349c1bf9af12bf8ec4fdf/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:32.497348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36e153-bda6-4e48-b7e6-31497db8b70a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:32.498053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36e153-bda6-4e48-b7e6-31497db8b70a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:32.498190 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36e153-bda6-4e48-b7e6-31497db8b70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:32.563503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9b36e153-bda6-4e48-b7e6-31497db8b70a None None] MFA Rules not processed for user `da571ebd580349c1bf9af12bf8ec4fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:32.565122 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9b36e153-bda6-4e48-b7e6-31497db8b70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:32.597471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd5257-2e70-4b0d-83ec-61adb8fa3c0c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:32.600453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-02bd5257-2e70-4b0d-83ec-61adb8fa3c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:32.765428 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:32.845052 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 684/1367] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:32.847629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36e153-bda6-4e48-b7e6-31497db8b70a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:32.860052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd5257-2e70-4b0d-83ec-61adb8fa3c0c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:32.870359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36e153-bda6-4e48-b7e6-31497db8b70a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:32.874915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:32.876089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:32.876577 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:32.877111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] PATH_INFO: `/v3/users/d96911960ec94380b902ea1d47bfa394` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:32.879177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96911960ec94380b902ea1d47bfa394)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:32.883105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:32.885020 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3f42e-a6c3-406a-8ac7-f74dbb1ac606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:32.887413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3f42e-a6c3-406a-8ac7-f74dbb1ac606 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:32.890216 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3f42e-a6c3-406a-8ac7-f74dbb1ac606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:32.891622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3f42e-a6c3-406a-8ac7-f74dbb1ac606 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:32.892948 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3f42e-a6c3-406a-8ac7-f74dbb1ac606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:32.895828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3f42e-a6c3-406a-8ac7-f74dbb1ac606 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:32.961971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 685/1368] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:32 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:32.968117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3f42e-a6c3-406a-8ac7-f74dbb1ac606 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:33.050036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96911960ec94380b902ea1d47bfa394'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:33.064050 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:33.099721 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96911960ec94380b902ea1d47bfa394'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:33.102868 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96911960ec94380b902ea1d47bfa394'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:33.130900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96911960ec94380b902ea1d47bfa394'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:33.133664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96911960ec94380b902ea1d47bfa394'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:33.178467 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 684/1369] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:32 2022] DELETE /identity/v3/users/d96911960ec94380b902ea1d47bfa394 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:33.226337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3f42e-a6c3-406a-8ac7-f74dbb1ac606 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:33.357790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7df05-a67d-4ad8-96e7-d9cda9df63d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:33.358447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7df05-a67d-4ad8-96e7-d9cda9df63d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:33.358940 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7df05-a67d-4ad8-96e7-d9cda9df63d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:33.359218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7df05-a67d-4ad8-96e7-d9cda9df63d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:33.359879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7df05-a67d-4ad8-96e7-d9cda9df63d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:33.361953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7df05-a67d-4ad8-96e7-d9cda9df63d9 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:33.410516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:33.412021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 686/1370] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:32 2022] POST /identity/v3/users => generated 297 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:40:33.412803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:33.433222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce439e8-f9d3-4b39-a52f-964712882cf4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:33.440097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e1acf-82e2-426d-9335-ebf071cf2ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:33.441434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e1acf-82e2-426d-9335-ebf071cf2ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:33.441875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e1acf-82e2-426d-9335-ebf071cf2ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:33.442157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e1acf-82e2-426d-9335-ebf071cf2ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:33.442832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e1acf-82e2-426d-9335-ebf071cf2ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:33.444634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e1acf-82e2-426d-9335-ebf071cf2ccb admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:33.468375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7df05-a67d-4ad8-96e7-d9cda9df63d9 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:33.472823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 685/1371] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:33.481006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7df05-a67d-4ad8-96e7-d9cda9df63d9 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:33.488857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a38e1-3e6e-43c0-be8f-eb56afec9cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:33.489753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a38e1-3e6e-43c0-be8f-eb56afec9cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:33.490035 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a38e1-3e6e-43c0-be8f-eb56afec9cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:33.490307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a38e1-3e6e-43c0-be8f-eb56afec9cd5 admin admin] PATH_INFO: `/v3/projects/e3990d114f5540bdaf141918421f1ad8/users/dd3b214588aa49c89f4d91c93546cf0d/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:33.538445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a38e1-3e6e-43c0-be8f-eb56afec9cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3990d114f5540bdaf141918421f1ad8, user_id=dd3b214588aa49c89f4d91c93546cf0d, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:33.543028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a38e1-3e6e-43c0-be8f-eb56afec9cd5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:33.600182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 687/1372] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:33 2022] PUT /identity/v3/projects/e3990d114f5540bdaf141918421f1ad8/users/dd3b214588aa49c89f4d91c93546cf0d/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:33.606681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e1acf-82e2-426d-9335-ebf071cf2ccb admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:33.623841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e1acf-82e2-426d-9335-ebf071cf2ccb admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:33.738860 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f065a-55a4-43f8-a607-89a9c50fbb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:33.740727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f065a-55a4-43f8-a607-89a9c50fbb37 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:33.741239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f065a-55a4-43f8-a607-89a9c50fbb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:33.741750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f065a-55a4-43f8-a607-89a9c50fbb37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:33.742642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f065a-55a4-43f8-a607-89a9c50fbb37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:33.744715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f065a-55a4-43f8-a607-89a9c50fbb37 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:33.767765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 686/1373] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:33.772017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a38e1-3e6e-43c0-be8f-eb56afec9cd5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:33.815678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a38e1-3e6e-43c0-be8f-eb56afec9cd5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:33.836597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb212f-78ef-4cc9-acad-9209e323d05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:33.838393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb212f-78ef-4cc9-acad-9209e323d05d admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:33.839620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb212f-78ef-4cc9-acad-9209e323d05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:33.841079 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb212f-78ef-4cc9-acad-9209e323d05d admin admin] PATH_INFO: `/v3/projects/e3990d114f5540bdaf141918421f1ad8/users/dd3b214588aa49c89f4d91c93546cf0d/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:33.844631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f065a-55a4-43f8-a607-89a9c50fbb37 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:33.846254 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb212f-78ef-4cc9-acad-9209e323d05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3990d114f5540bdaf141918421f1ad8, user_id=dd3b214588aa49c89f4d91c93546cf0d, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:33.847512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-da6f065a-55a4-43f8-a607-89a9c50fbb37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:33.851541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb212f-78ef-4cc9-acad-9209e323d05d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:33.867898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f065a-55a4-43f8-a607-89a9c50fbb37 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:33.879636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 688/1374] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:33 2022] PUT /identity/v3/projects/e3990d114f5540bdaf141918421f1ad8/users/dd3b214588aa49c89f4d91c93546cf0d/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:33.895939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e12c5-c562-4a89-a521-6fab1496ba3f None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:33.896239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e12c5-c562-4a89-a521-6fab1496ba3f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:33.896523 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e12c5-c562-4a89-a521-6fab1496ba3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:33.970262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-f27e12c5-c562-4a89-a521-6fab1496ba3f None None] MFA Rules not processed for user `dd3b214588aa49c89f4d91c93546cf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:33.971522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f27e12c5-c562-4a89-a521-6fab1496ba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:33.974548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:34.004566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:34.005313 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:34.005674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:34.005957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] PATH_INFO: `/v3/projects/b6e769716dc4464bb37102f6f6dcd033` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:34.007493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6e769716dc4464bb37102f6f6dcd033)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:34.012455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:34.091218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:34.102192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 689/1375] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:34.104583 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41186713-4fa4-402e-a694-e5d63dc15539 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:34.105299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41186713-4fa4-402e-a694-e5d63dc15539 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:34.105898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41186713-4fa4-402e-a694-e5d63dc15539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:34.142001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6e769716dc4464bb37102f6f6dcd033', 'request_id': 'req-2d146e5b-6d40-4e4a-a86d-5deeab02537e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:34.157554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6e769716dc4464bb37102f6f6dcd033', 'request_id': 'req-2d146e5b-6d40-4e4a-a86d-5deeab02537e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:34.162668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-41186713-4fa4-402e-a694-e5d63dc15539 None None] MFA Rules not processed for user `1f624c36a3574b61bd17d5d73af71916`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:34.164127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-41186713-4fa4-402e-a694-e5d63dc15539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:34.190099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6e769716dc4464bb37102f6f6dcd033', 'request_id': 'req-2d146e5b-6d40-4e4a-a86d-5deeab02537e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:34.218780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] The token cache is being invalidate because project b6e769716dc4464bb37102f6f6dcd033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:34.219271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:34.221241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:34.227636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 687/1376] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:33 2022] DELETE /identity/v3/projects/b6e769716dc4464bb37102f6f6dcd033 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:34.227636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:34.232179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:34.311069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 690/1377] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:34.323109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41186713-4fa4-402e-a694-e5d63dc15539 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:34.324668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-41186713-4fa4-402e-a694-e5d63dc15539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:34.407798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d146e5b-6d40-4e4a-a86d-5deeab02537e admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:34.417017 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013e8ab-f45f-43c0-a15b-5d1b6d984c98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:34.417827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e8ab-f45f-43c0-a15b-5d1b6d984c98 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:34.418215 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e8ab-f45f-43c0-a15b-5d1b6d984c98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:34.419166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e8ab-f45f-43c0-a15b-5d1b6d984c98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:34.421938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4013e8ab-f45f-43c0-a15b-5d1b6d984c98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:34.494396 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41186713-4fa4-402e-a694-e5d63dc15539 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:34.512920 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:34.513601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:34.513877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:34.514147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] PATH_INFO: `/v3/users/396ca9ae07024ceb99e877995910e4cb` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:34.515669 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=396ca9ae07024ceb99e877995910e4cb)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:34.517688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:34.554602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013e8ab-f45f-43c0-a15b-5d1b6d984c98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:34.556092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013e8ab-f45f-43c0-a15b-5d1b6d984c98 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:34.576927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 688/1378] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:34.581326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e404d-26bc-45ed-9c58-3ddd4d816219 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:34.581629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e404d-26bc-45ed-9c58-3ddd4d816219 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:34.581901 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e404d-26bc-45ed-9c58-3ddd4d816219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:34.629382 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-f52e404d-26bc-45ed-9c58-3ddd4d816219 None None] MFA Rules not processed for user `1f624c36a3574b61bd17d5d73af71916`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:34.630978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f52e404d-26bc-45ed-9c58-3ddd4d816219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:34.652334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 689/1379] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:34.652334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2885-e55d-46c4-8c36-b2901c7ee131 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:34.652334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2885-e55d-46c4-8c36-b2901c7ee131 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:34.652334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e2885-e55d-46c4-8c36-b2901c7ee131 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:34.652334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 690/1380] 10.222.0.17 () {54 vars in 912 bytes} [Tue Oct 18 11:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:40:34.657498 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf4427-1920-4979-abf8-f638930ac881 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:34.657787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf4427-1920-4979-abf8-f638930ac881 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:34.658078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf4427-1920-4979-abf8-f638930ac881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:34.702231 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '396ca9ae07024ceb99e877995910e4cb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:34.728787 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '396ca9ae07024ceb99e877995910e4cb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:34.731938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '396ca9ae07024ceb99e877995910e4cb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:34.743041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:34.795409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-cbaf4427-1920-4979-abf8-f638930ac881 None None] MFA Rules not processed for user `d301f3d9aeb644a38c31dbdbef2a48c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:34.798171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cbaf4427-1920-4979-abf8-f638930ac881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:34.798926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '396ca9ae07024ceb99e877995910e4cb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:34.815021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '396ca9ae07024ceb99e877995910e4cb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:34.827212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 691/1381] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:34 2022] DELETE /identity/v3/users/396ca9ae07024ceb99e877995910e4cb => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:34.828128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:34.829591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:34.866444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:34.967829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 691/1382] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:40:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:40:34.970513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf4427-1920-4979-abf8-f638930ac881 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:34.972570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cbaf4427-1920-4979-abf8-f638930ac881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:35.009492 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf4427-1920-4979-abf8-f638930ac881 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:35.054754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d560-1be4-4002-b898-e9d28c46e094 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:35.073553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b20403-099e-486d-8d3f-6d09b0bfb5e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:35.074199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20403-099e-486d-8d3f-6d09b0bfb5e6 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:35.074479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20403-099e-486d-8d3f-6d09b0bfb5e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:35.074800 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b20403-099e-486d-8d3f-6d09b0bfb5e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:35.078224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-39b20403-099e-486d-8d3f-6d09b0bfb5e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:35.185258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24584ee-d8f7-4904-a700-9b4e90169641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:35.188973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24584ee-d8f7-4904-a700-9b4e90169641 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:35.189464 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24584ee-d8f7-4904-a700-9b4e90169641 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:35.190540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24584ee-d8f7-4904-a700-9b4e90169641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:35.197735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f24584ee-d8f7-4904-a700-9b4e90169641 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:35.333528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b20403-099e-486d-8d3f-6d09b0bfb5e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:35.335518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b20403-099e-486d-8d3f-6d09b0bfb5e6 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:35.391466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 692/1383] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:35.392349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b20403-099e-486d-8d3f-6d09b0bfb5e6 service nova] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:35.393744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-39b20403-099e-486d-8d3f-6d09b0bfb5e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:35.429334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24584ee-d8f7-4904-a700-9b4e90169641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:35.432132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24584ee-d8f7-4904-a700-9b4e90169641 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:35.448698 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 692/1384] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:35.449907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24584ee-d8f7-4904-a700-9b4e90169641 service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:35.451765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f24584ee-d8f7-4904-a700-9b4e90169641 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:35.557941 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24584ee-d8f7-4904-a700-9b4e90169641 service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:35.564231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f718a2-8ccb-4ef3-b51d-20135d0b609a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:35.565154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f718a2-8ccb-4ef3-b51d-20135d0b609a service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:35.565446 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f718a2-8ccb-4ef3-b51d-20135d0b609a service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:35.566218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f718a2-8ccb-4ef3-b51d-20135d0b609a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:35.581795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f718a2-8ccb-4ef3-b51d-20135d0b609a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:35.583554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f718a2-8ccb-4ef3-b51d-20135d0b609a service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:35.609663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 693/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:35.614094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b20403-099e-486d-8d3f-6d09b0bfb5e6 service nova] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:35.631430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:35.632194 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:35.632477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:35.632771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] PATH_INFO: `/v3/projects/c4144984961a4ec78c81659ce0ca5e9e` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:35.634042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4144984961a4ec78c81659ce0ca5e9e)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:35.635069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: WARNING py.warnings [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 18 11:40:35.635069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: warnings.warn(msg) Oct 18 11:40:35.635069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: Oct 18 11:40:35.636510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:35.801760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:35.807636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4144984961a4ec78c81659ce0ca5e9e', 'request_id': 'req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:35.826272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4144984961a4ec78c81659ce0ca5e9e', 'request_id': 'req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:35.839257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4144984961a4ec78c81659ce0ca5e9e', 'request_id': 'req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:35.866553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] The token cache is being invalidate because project c4144984961a4ec78c81659ce0ca5e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:35.867194 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:35.868511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-0686219e-7954-4fbe-9c2f-9f4ccffbcf16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:35.871299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 693/1386] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:35 2022] DELETE /identity/v3/projects/c4144984961a4ec78c81659ce0ca5e9e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:35.875981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f718a2-8ccb-4ef3-b51d-20135d0b609a service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:35.900967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f718a2-8ccb-4ef3-b51d-20135d0b609a service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:35.912240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:36.014465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:36.015341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:36.015683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:36.015968 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] PATH_INFO: `/v3/users/8c5dd927177443dba4eeb820c318b2f0` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:36.017298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c5dd927177443dba4eeb820c318b2f0)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:36.019664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:36.122053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c5dd927177443dba4eeb820c318b2f0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:36.158573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c5dd927177443dba4eeb820c318b2f0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:36.159400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c5dd927177443dba4eeb820c318b2f0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:36.193540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c5dd927177443dba4eeb820c318b2f0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:36.194321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8e065901-2d7e-4282-915c-0194bd390f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c5dd927177443dba4eeb820c318b2f0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:36.218245 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 694/1387] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:35 2022] DELETE /identity/v3/users/8c5dd927177443dba4eeb820c318b2f0 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:36.463636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba786610-f529-4574-bb8b-7cb4159d3119 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:36.463973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba786610-f529-4574-bb8b-7cb4159d3119 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:36.464297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba786610-f529-4574-bb8b-7cb4159d3119 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:36.466265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 694/1388] 10.222.0.17 () {54 vars in 912 bytes} [Tue Oct 18 11:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:40:36.476353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fecd1-f6f9-42d0-856f-aaf95eac284a None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:36.476642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fecd1-f6f9-42d0-856f-aaf95eac284a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:36.476948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fecd1-f6f9-42d0-856f-aaf95eac284a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:36.516318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba786610-f529-4574-bb8b-7cb4159d3119 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:36.518151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ba786610-f529-4574-bb8b-7cb4159d3119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:36.532942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-cd6fecd1-f6f9-42d0-856f-aaf95eac284a None None] MFA Rules not processed for user `d301f3d9aeb644a38c31dbdbef2a48c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:36.535964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cd6fecd1-f6f9-42d0-856f-aaf95eac284a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:36.665294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 695/1389] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:40:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:36.666766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b2757-ea35-4f77-8c68-edbf251a7a35 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:36.667723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b2757-ea35-4f77-8c68-edbf251a7a35 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:36.667723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b2757-ea35-4f77-8c68-edbf251a7a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:36.751177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-0f4b2757-ea35-4f77-8c68-edbf251a7a35 None None] MFA Rules not processed for user `916699a212eb4d8aa945f299ac0d4c96`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:36.753621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0f4b2757-ea35-4f77-8c68-edbf251a7a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:36.792246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba786610-f529-4574-bb8b-7cb4159d3119 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:36.800773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:36.802276 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:36.802929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:36.803382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] PATH_INFO: `/v3/projects/85a6a15728fa45058b592d7802531219` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:36.805437 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a6a15728fa45058b592d7802531219)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:36.808237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:36.888174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:36.963676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 696/1390] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:36.966361 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed903f-f302-45b4-819e-e6dd5e85abbf None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:36.966673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed903f-f302-45b4-819e-e6dd5e85abbf None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:36.967326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed903f-f302-45b4-819e-e6dd5e85abbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:36.992146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-21ed903f-f302-45b4-819e-e6dd5e85abbf None None] MFA Rules not processed for user `bddd3d2bb8f647f2ba616141eb16622e`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:36.993545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-21ed903f-f302-45b4-819e-e6dd5e85abbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:37.103706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 697/1391] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:37.104692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed903f-f302-45b4-819e-e6dd5e85abbf None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:37.107341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-21ed903f-f302-45b4-819e-e6dd5e85abbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:37.119349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a6a15728fa45058b592d7802531219', 'request_id': 'req-cb75c520-be21-484b-8889-17adb8148cf5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:37.135638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a6a15728fa45058b592d7802531219', 'request_id': 'req-cb75c520-be21-484b-8889-17adb8148cf5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:37.146191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed903f-f302-45b4-819e-e6dd5e85abbf None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:37.154986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a6a15728fa45058b592d7802531219', 'request_id': 'req-cb75c520-be21-484b-8889-17adb8148cf5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:37.193270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] The token cache is being invalidate because project 85a6a15728fa45058b592d7802531219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:37.193814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:37.194693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:37.196511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 695/1392] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:36 2022] DELETE /identity/v3/projects/85a6a15728fa45058b592d7802531219 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:40:37.197094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:37.198514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:37.274653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49691c66-b3bf-4058-9ada-30a2682a2853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:37.275283 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49691c66-b3bf-4058-9ada-30a2682a2853 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:37.275564 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49691c66-b3bf-4058-9ada-30a2682a2853 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:37.275835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49691c66-b3bf-4058-9ada-30a2682a2853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:37.284652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-49691c66-b3bf-4058-9ada-30a2682a2853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:37.298423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49691c66-b3bf-4058-9ada-30a2682a2853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:37.299932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49691c66-b3bf-4058-9ada-30a2682a2853 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:37.319168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75c520-be21-484b-8889-17adb8148cf5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:37.321250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 698/1393] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:37.322119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49691c66-b3bf-4058-9ada-30a2682a2853 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:37.332390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-49691c66-b3bf-4058-9ada-30a2682a2853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:37.336765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58900a8f-83f9-4c0e-85ac-15d288d4a9ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:37.337721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58900a8f-83f9-4c0e-85ac-15d288d4a9ef service placement] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:37.338105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58900a8f-83f9-4c0e-85ac-15d288d4a9ef service placement] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:37.338495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58900a8f-83f9-4c0e-85ac-15d288d4a9ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:37.341169 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-58900a8f-83f9-4c0e-85ac-15d288d4a9ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:37.378963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:37.384931 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58900a8f-83f9-4c0e-85ac-15d288d4a9ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:37.392800 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58900a8f-83f9-4c0e-85ac-15d288d4a9ef service placement] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:37.427014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 696/1394] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:37.428499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58900a8f-83f9-4c0e-85ac-15d288d4a9ef service placement] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:37.431643 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-58900a8f-83f9-4c0e-85ac-15d288d4a9ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:37.519522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49691c66-b3bf-4058-9ada-30a2682a2853 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:37.546119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:37.546450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:37.546734 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:37.547009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] PATH_INFO: `/v3/projects/625198a0d2734fc49ca2a6712a6bea07` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:37.549927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=625198a0d2734fc49ca2a6712a6bea07)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:37.551832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:37.619724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:37.641330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58900a8f-83f9-4c0e-85ac-15d288d4a9ef service placement] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:37.664930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb4cb3-faa6-48d0-be56-87d1937c8e9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:37.665602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb4cb3-faa6-48d0-be56-87d1937c8e9f service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:37.666182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb4cb3-faa6-48d0-be56-87d1937c8e9f service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:37.666508 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb4cb3-faa6-48d0-be56-87d1937c8e9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:37.670136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3aeb4cb3-faa6-48d0-be56-87d1937c8e9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:37.691528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625198a0d2734fc49ca2a6712a6bea07', 'request_id': 'req-0f4de4fa-09a7-444c-96b6-47cc2cae4914'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:37.701541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625198a0d2734fc49ca2a6712a6bea07', 'request_id': 'req-0f4de4fa-09a7-444c-96b6-47cc2cae4914'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:37.721768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625198a0d2734fc49ca2a6712a6bea07', 'request_id': 'req-0f4de4fa-09a7-444c-96b6-47cc2cae4914'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:37.759746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] The token cache is being invalidate because project 625198a0d2734fc49ca2a6712a6bea07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:37.760045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:37.763877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f4de4fa-09a7-444c-96b6-47cc2cae4914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:37.766952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 699/1395] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:37 2022] DELETE /identity/v3/projects/625198a0d2734fc49ca2a6712a6bea07 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:37.769023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4234f-9e8e-4783-94f7-075fccad3e16 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:37.770495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4234f-9e8e-4783-94f7-075fccad3e16 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:37.770894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4234f-9e8e-4783-94f7-075fccad3e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:37.804834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb4cb3-faa6-48d0-be56-87d1937c8e9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:37.810761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb4cb3-faa6-48d0-be56-87d1937c8e9f service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:37.815140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6da4234f-9e8e-4783-94f7-075fccad3e16 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:37.816627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6da4234f-9e8e-4783-94f7-075fccad3e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:37.842226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 697/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:37.843644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6b405-7265-4a4f-a73c-4fa6d382d0b8 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:37.857803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6b405-7265-4a4f-a73c-4fa6d382d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:37.858224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6b405-7265-4a4f-a73c-4fa6d382d0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:37.937324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-57e6b405-7265-4a4f-a73c-4fa6d382d0b8 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:37.944333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-57e6b405-7265-4a4f-a73c-4fa6d382d0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:37.952486 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 700/1397] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:37.960112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf4fb5-1a13-4114-8944-461a2e9b3b38 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:37.960368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf4fb5-1a13-4114-8944-461a2e9b3b38 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:37.960647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf4fb5-1a13-4114-8944-461a2e9b3b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:37.973926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 698/1398] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:37.982734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f75c4d-4824-4e47-a120-992c131c4801 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:37.983081 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f75c4d-4824-4e47-a120-992c131c4801 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:37.983361 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f75c4d-4824-4e47-a120-992c131c4801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.033449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e3cf4fb5-1a13-4114-8944-461a2e9b3b38 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:38.034894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e3cf4fb5-1a13-4114-8944-461a2e9b3b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:38.048619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-98f75c4d-4824-4e47-a120-992c131c4801 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:38.050605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-98f75c4d-4824-4e47-a120-992c131c4801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:38.054036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 701/1399] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:38.061747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf4fb5-1a13-4114-8944-461a2e9b3b38 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.089446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 699/1400] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:38.092281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f75c4d-4824-4e47-a120-992c131c4801 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.097569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf4fb5-1a13-4114-8944-461a2e9b3b38 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.118538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf09a9-c742-4117-ab6b-489fbf5e0298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.121317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf09a9-c742-4117-ab6b-489fbf5e0298 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.123731 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f75c4d-4824-4e47-a120-992c131c4801 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.124353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf09a9-c742-4117-ab6b-489fbf5e0298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.124641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf09a9-c742-4117-ab6b-489fbf5e0298 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.126905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf09a9-c742-4117-ab6b-489fbf5e0298 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:38.127214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf09a9-c742-4117-ab6b-489fbf5e0298 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.129813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf09a9-c742-4117-ab6b-489fbf5e0298 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.142911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941d63d-03f5-406e-a400-c1b2b53fba38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.143847 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941d63d-03f5-406e-a400-c1b2b53fba38 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.143847 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941d63d-03f5-406e-a400-c1b2b53fba38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.143847 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941d63d-03f5-406e-a400-c1b2b53fba38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.148832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941d63d-03f5-406e-a400-c1b2b53fba38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:38.149164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941d63d-03f5-406e-a400-c1b2b53fba38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.157356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941d63d-03f5-406e-a400-c1b2b53fba38 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.185525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 702/1401] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:38.189224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf09a9-c742-4117-ab6b-489fbf5e0298 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.212913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 700/1402] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:38.217529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941d63d-03f5-406e-a400-c1b2b53fba38 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.222261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf09a9-c742-4117-ab6b-489fbf5e0298 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.236392 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f81a6-11a5-41de-8339-b73dd71a45f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.238998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f81a6-11a5-41de-8339-b73dd71a45f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.239897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f81a6-11a5-41de-8339-b73dd71a45f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.240274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f81a6-11a5-41de-8339-b73dd71a45f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.244614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941d63d-03f5-406e-a400-c1b2b53fba38 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.245400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f81a6-11a5-41de-8339-b73dd71a45f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.268051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f81a6-11a5-41de-8339-b73dd71a45f0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.277684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d25a99-0843-40ff-9862-7ca9131929b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.278183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d25a99-0843-40ff-9862-7ca9131929b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.280363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d25a99-0843-40ff-9862-7ca9131929b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.280660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d25a99-0843-40ff-9862-7ca9131929b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.284394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d25a99-0843-40ff-9862-7ca9131929b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.288409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d25a99-0843-40ff-9862-7ca9131929b3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.351952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 703/1403] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:38 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:38.357432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f81a6-11a5-41de-8339-b73dd71a45f0 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.373127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 701/1404] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:38 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:38.374022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d25a99-0843-40ff-9862-7ca9131929b3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.497853 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:38.522266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f81a6-11a5-41de-8339-b73dd71a45f0 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.539432 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d25a99-0843-40ff-9862-7ca9131929b3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.544019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccb4d0-a8e7-48b6-8231-e9d1ba1e0bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.545185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccb4d0-a8e7-48b6-8231-e9d1ba1e0bdc admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.545671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccb4d0-a8e7-48b6-8231-e9d1ba1e0bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.546945 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccb4d0-a8e7-48b6-8231-e9d1ba1e0bdc admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.547866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccb4d0-a8e7-48b6-8231-e9d1ba1e0bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.549692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7ae06-13d0-4198-beea-1a8c8301f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.550314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ae06-13d0-4198-beea-1a8c8301f69e admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.550596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ae06-13d0-4198-beea-1a8c8301f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.550871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ae06-13d0-4198-beea-1a8c8301f69e admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.551181 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccb4d0-a8e7-48b6-8231-e9d1ba1e0bdc admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.551632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7ae06-13d0-4198-beea-1a8c8301f69e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.554247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7ae06-13d0-4198-beea-1a8c8301f69e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.597732 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:38.617300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 702/1405] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:38 2022] POST /identity/v3/users => generated 287 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:38.625864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7ae06-13d0-4198-beea-1a8c8301f69e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.626221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 704/1406] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:38 2022] POST /identity/v3/users => generated 291 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:40:38.630767 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccb4d0-a8e7-48b6-8231-e9d1ba1e0bdc admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.643970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccb4d0-a8e7-48b6-8231-e9d1ba1e0bdc admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.651111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea293614-f90c-48c6-86ee-b4e60525c158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.651111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea293614-f90c-48c6-86ee-b4e60525c158 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.651418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea293614-f90c-48c6-86ee-b4e60525c158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.651418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea293614-f90c-48c6-86ee-b4e60525c158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.651539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7ae06-13d0-4198-beea-1a8c8301f69e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.652246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea293614-f90c-48c6-86ee-b4e60525c158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.654146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea293614-f90c-48c6-86ee-b4e60525c158 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.672375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1d51e-90c9-4715-ae50-53186b975eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.673041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1d51e-90c9-4715-ae50-53186b975eea admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.673323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1d51e-90c9-4715-ae50-53186b975eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.673648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1d51e-90c9-4715-ae50-53186b975eea admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.674308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1d51e-90c9-4715-ae50-53186b975eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.675994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1d51e-90c9-4715-ae50-53186b975eea admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.699589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 703/1407] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:38.700591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1d51e-90c9-4715-ae50-53186b975eea admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.704124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 705/1408] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:38.707457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea293614-f90c-48c6-86ee-b4e60525c158 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.727037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1d51e-90c9-4715-ae50-53186b975eea admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.735820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2274e4-7d8b-4d76-8237-08d4faae95db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.736533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2274e4-7d8b-4d76-8237-08d4faae95db admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.736790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2274e4-7d8b-4d76-8237-08d4faae95db admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.737008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea293614-f90c-48c6-86ee-b4e60525c158 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.737090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2274e4-7d8b-4d76-8237-08d4faae95db admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.737740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2274e4-7d8b-4d76-8237-08d4faae95db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.739428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2274e4-7d8b-4d76-8237-08d4faae95db admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.752683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab524d4-57d9-4aca-8df0-afc13ed1de4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.753356 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab524d4-57d9-4aca-8df0-afc13ed1de4b admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.753638 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab524d4-57d9-4aca-8df0-afc13ed1de4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.758658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab524d4-57d9-4aca-8df0-afc13ed1de4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.761099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab524d4-57d9-4aca-8df0-afc13ed1de4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.767293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 704/1409] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:38.767574 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab524d4-57d9-4aca-8df0-afc13ed1de4b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.771816 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2274e4-7d8b-4d76-8237-08d4faae95db admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.789610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2274e4-7d8b-4d76-8237-08d4faae95db admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.801323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6630d66-4d7d-48c2-9520-9aacf2ecc195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.802128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6630d66-4d7d-48c2-9520-9aacf2ecc195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.802540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6630d66-4d7d-48c2-9520-9aacf2ecc195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.802925 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6630d66-4d7d-48c2-9520-9aacf2ecc195 admin admin] PATH_INFO: `/v3/projects/862d049d044e442391fbd4bf36db3414/users/4c1a459b89de421889d41a0c3bdb037e/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.803384 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 706/1410] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:38.824328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab524d4-57d9-4aca-8df0-afc13ed1de4b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:38.838390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab524d4-57d9-4aca-8df0-afc13ed1de4b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:38.846160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd3c43-0250-4d68-8f57-3fa3020ac120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:38.847607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd3c43-0250-4d68-8f57-3fa3020ac120 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.848097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd3c43-0250-4d68-8f57-3fa3020ac120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.848632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd3c43-0250-4d68-8f57-3fa3020ac120 admin admin] PATH_INFO: `/v3/projects/60dc23e43cc34c39a77700b1d329ce81/users/825d9df94a18454d99304eea083b03a8/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.851509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6630d66-4d7d-48c2-9520-9aacf2ecc195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862d049d044e442391fbd4bf36db3414, user_id=4c1a459b89de421889d41a0c3bdb037e, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.865298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6630d66-4d7d-48c2-9520-9aacf2ecc195 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.869760 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd3c43-0250-4d68-8f57-3fa3020ac120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60dc23e43cc34c39a77700b1d329ce81, user_id=825d9df94a18454d99304eea083b03a8, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:38.872782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd3c43-0250-4d68-8f57-3fa3020ac120 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:38.898475 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 705/1411] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:38 2022] PUT /identity/v3/projects/862d049d044e442391fbd4bf36db3414/users/4c1a459b89de421889d41a0c3bdb037e/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:40:38.912315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096b5e2-8d2c-4e9a-9439-83452f0e61c6 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.913953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096b5e2-8d2c-4e9a-9439-83452f0e61c6 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.916619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096b5e2-8d2c-4e9a-9439-83452f0e61c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:38.929254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 707/1412] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:38 2022] PUT /identity/v3/projects/60dc23e43cc34c39a77700b1d329ce81/users/825d9df94a18454d99304eea083b03a8/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:38.937282 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a9ddb-6460-4be1-98b7-0c974beb472c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:38.937571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a9ddb-6460-4be1-98b7-0c974beb472c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:38.937848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a9ddb-6460-4be1-98b7-0c974beb472c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:39.019923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4096b5e2-8d2c-4e9a-9439-83452f0e61c6 None None] MFA Rules not processed for user `4c1a459b89de421889d41a0c3bdb037e`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:39.031277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d91a9ddb-6460-4be1-98b7-0c974beb472c None None] MFA Rules not processed for user `825d9df94a18454d99304eea083b03a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:39.032713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d91a9ddb-6460-4be1-98b7-0c974beb472c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:39.034261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4096b5e2-8d2c-4e9a-9439-83452f0e61c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:39.247675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 708/1413] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:39.247675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fa3c2-e70e-44f8-bdd9-3c4f828973e1 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:39.247675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fa3c2-e70e-44f8-bdd9-3c4f828973e1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:39.247675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fa3c2-e70e-44f8-bdd9-3c4f828973e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:39.255913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 706/1414] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:39.257119 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096b5e2-8d2c-4e9a-9439-83452f0e61c6 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:39.337288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-0c2fa3c2-e70e-44f8-bdd9-3c4f828973e1 None None] MFA Rules not processed for user `1f624c36a3574b61bd17d5d73af71916`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:39.357159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0c2fa3c2-e70e-44f8-bdd9-3c4f828973e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:39.426594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096b5e2-8d2c-4e9a-9439-83452f0e61c6 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:39.440650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631a53e-1283-41fc-bcf6-20d7b25f2d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:39.441516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a53e-1283-41fc-bcf6-20d7b25f2d10 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:39.442184 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a53e-1283-41fc-bcf6-20d7b25f2d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:39.442184 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631a53e-1283-41fc-bcf6-20d7b25f2d10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:39.442703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631a53e-1283-41fc-bcf6-20d7b25f2d10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:39.444630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631a53e-1283-41fc-bcf6-20d7b25f2d10 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:39.518065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 707/1415] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:39 2022] POST /identity/v3/projects => generated 373 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:39.523335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631a53e-1283-41fc-bcf6-20d7b25f2d10 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:39.535859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:39.593453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 709/1416] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:39.594450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fa3c2-e70e-44f8-bdd9-3c4f828973e1 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:39.664681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fa3c2-e70e-44f8-bdd9-3c4f828973e1 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:39.770790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631a53e-1283-41fc-bcf6-20d7b25f2d10 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:39.787332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc143ddd-f209-4ba4-8f2d-5a9107b38b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:39.788187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc143ddd-f209-4ba4-8f2d-5a9107b38b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:39.788610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc143ddd-f209-4ba4-8f2d-5a9107b38b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:39.789065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc143ddd-f209-4ba4-8f2d-5a9107b38b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:39.789953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc143ddd-f209-4ba4-8f2d-5a9107b38b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:39.794319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc143ddd-f209-4ba4-8f2d-5a9107b38b75 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:39.855621 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 708/1417] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:39 2022] POST /identity/v3/projects => generated 379 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:39.861873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc143ddd-f209-4ba4-8f2d-5a9107b38b75 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:39.863801 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:39.928307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66c761-3ba2-4eae-8504-1fa9d7922f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:39.929090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c761-3ba2-4eae-8504-1fa9d7922f34 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:39.929376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c761-3ba2-4eae-8504-1fa9d7922f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:39.929655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c761-3ba2-4eae-8504-1fa9d7922f34 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:39.932092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66c761-3ba2-4eae-8504-1fa9d7922f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:39.936682 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66c761-3ba2-4eae-8504-1fa9d7922f34 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:40.017742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 710/1418] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:39 2022] POST /identity/v3/users => generated 293 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:40.022468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66c761-3ba2-4eae-8504-1fa9d7922f34 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:40.048810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc143ddd-f209-4ba4-8f2d-5a9107b38b75 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:40.064698 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c224a-7c41-4b70-acee-560dbb3b8cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:40.065004 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66c761-3ba2-4eae-8504-1fa9d7922f34 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:40.069697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c224a-7c41-4b70-acee-560dbb3b8cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:40.070088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c224a-7c41-4b70-acee-560dbb3b8cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:40.070371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c224a-7c41-4b70-acee-560dbb3b8cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:40.071084 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c224a-7c41-4b70-acee-560dbb3b8cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:40.080070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c224a-7c41-4b70-acee-560dbb3b8cb2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:40.083623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dd58a-926b-4526-a03e-0973e0cbe510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:40.084470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dd58a-926b-4526-a03e-0973e0cbe510 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:40.084756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dd58a-926b-4526-a03e-0973e0cbe510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:40.085092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dd58a-926b-4526-a03e-0973e0cbe510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:40.085762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dd58a-926b-4526-a03e-0973e0cbe510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:40.087390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dd58a-926b-4526-a03e-0973e0cbe510 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:40.130294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 711/1419] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:40.133212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dd58a-926b-4526-a03e-0973e0cbe510 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:40.155040 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 709/1420] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:39 2022] POST /identity/v3/users => generated 296 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:40.180630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c224a-7c41-4b70-acee-560dbb3b8cb2 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:40.188588 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dd58a-926b-4526-a03e-0973e0cbe510 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:40.207693 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48dcd5-e745-49f8-a72d-17be149cb7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:40.208395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48dcd5-e745-49f8-a72d-17be149cb7b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:40.213872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48dcd5-e745-49f8-a72d-17be149cb7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:40.213872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48dcd5-e745-49f8-a72d-17be149cb7b6 admin admin] PATH_INFO: `/v3/projects/950c2a3d60ac432cb8a76d53df60b0a4/users/978f4c97d5a74171a3466e6b33e032a8/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:40.233968 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c224a-7c41-4b70-acee-560dbb3b8cb2 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:40.249255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710b19-2f2a-4ea2-b0fd-826cbe642d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:40.249467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710b19-2f2a-4ea2-b0fd-826cbe642d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:40.256565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710b19-2f2a-4ea2-b0fd-826cbe642d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:40.256985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710b19-2f2a-4ea2-b0fd-826cbe642d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:40.257714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81710b19-2f2a-4ea2-b0fd-826cbe642d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:40.265345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81710b19-2f2a-4ea2-b0fd-826cbe642d34 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:40.273008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48dcd5-e745-49f8-a72d-17be149cb7b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=950c2a3d60ac432cb8a76d53df60b0a4, user_id=978f4c97d5a74171a3466e6b33e032a8, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:40.276488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48dcd5-e745-49f8-a72d-17be149cb7b6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:40.311227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 712/1421] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:40 2022] PUT /identity/v3/projects/950c2a3d60ac432cb8a76d53df60b0a4/users/978f4c97d5a74171a3466e6b33e032a8/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:40.315210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48dcd5-e745-49f8-a72d-17be149cb7b6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:40.327239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 710/1422] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:40.328236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710b19-2f2a-4ea2-b0fd-826cbe642d34 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:40.390806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710b19-2f2a-4ea2-b0fd-826cbe642d34 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:40.545506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48dcd5-e745-49f8-a72d-17be149cb7b6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:40.561031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba161ca7-529f-4d8c-b5bb-d1464bb5c521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:40.562097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba161ca7-529f-4d8c-b5bb-d1464bb5c521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:40.562596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba161ca7-529f-4d8c-b5bb-d1464bb5c521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:40.563086 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba161ca7-529f-4d8c-b5bb-d1464bb5c521 admin admin] PATH_INFO: `/v3/projects/9a40cec64c314b4a90d5ed15003185c8/users/d4ce4a6f117b418994844191999f7c42/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:40.575170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08f637-ca59-48e7-97c7-b2ecc1190012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:40.576244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08f637-ca59-48e7-97c7-b2ecc1190012 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:40.576531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08f637-ca59-48e7-97c7-b2ecc1190012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:40.576797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08f637-ca59-48e7-97c7-b2ecc1190012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:40.577499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08f637-ca59-48e7-97c7-b2ecc1190012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:40.579127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08f637-ca59-48e7-97c7-b2ecc1190012 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:40.584149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:40.601047 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba161ca7-529f-4d8c-b5bb-d1464bb5c521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a40cec64c314b4a90d5ed15003185c8, user_id=d4ce4a6f117b418994844191999f7c42, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:40.610373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba161ca7-529f-4d8c-b5bb-d1464bb5c521 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:40.622947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 713/1423] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:40.626328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08f637-ca59-48e7-97c7-b2ecc1190012 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:40.646104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 711/1424] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:40 2022] PUT /identity/v3/projects/9a40cec64c314b4a90d5ed15003185c8/users/d4ce4a6f117b418994844191999f7c42/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:40.650312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba161ca7-529f-4d8c-b5bb-d1464bb5c521 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:40.653210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08f637-ca59-48e7-97c7-b2ecc1190012 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:40.822807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba161ca7-529f-4d8c-b5bb-d1464bb5c521 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:40.839691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc54f68-2315-4c98-9556-716c98cc6663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:40.847108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc54f68-2315-4c98-9556-716c98cc6663 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:40.847658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc54f68-2315-4c98-9556-716c98cc6663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:40.848040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc54f68-2315-4c98-9556-716c98cc6663 admin admin] PATH_INFO: `/v3/projects/950c2a3d60ac432cb8a76d53df60b0a4/users/978f4c97d5a74171a3466e6b33e032a8/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:40.849175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d94bf-0eef-4805-8ad4-2d369228fe74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:40.850338 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d94bf-0eef-4805-8ad4-2d369228fe74 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:40.850641 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d94bf-0eef-4805-8ad4-2d369228fe74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:40.850919 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d94bf-0eef-4805-8ad4-2d369228fe74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:40.851591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d94bf-0eef-4805-8ad4-2d369228fe74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:40.853400 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d94bf-0eef-4805-8ad4-2d369228fe74 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:40.854434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc54f68-2315-4c98-9556-716c98cc6663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=950c2a3d60ac432cb8a76d53df60b0a4, user_id=978f4c97d5a74171a3466e6b33e032a8, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:40.857840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc54f68-2315-4c98-9556-716c98cc6663 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:40.884067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 714/1425] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:40 2022] PUT /identity/v3/projects/950c2a3d60ac432cb8a76d53df60b0a4/users/978f4c97d5a74171a3466e6b33e032a8/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:40:40.896038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f4bcc-0305-4b94-a670-efb8ad4b1f54 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:40.896476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f4bcc-0305-4b94-a670-efb8ad4b1f54 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:40.896998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f4bcc-0305-4b94-a670-efb8ad4b1f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:40.915843 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 712/1426] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:40.920608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d94bf-0eef-4805-8ad4-2d369228fe74 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:40.965937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d94bf-0eef-4805-8ad4-2d369228fe74 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:40.976480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-de9f4bcc-0305-4b94-a670-efb8ad4b1f54 None None] MFA Rules not processed for user `978f4c97d5a74171a3466e6b33e032a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:40.977759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-de9f4bcc-0305-4b94-a670-efb8ad4b1f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:41.052339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:41.148179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c836c-59e8-4902-8836-1e073d07e5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:41.149843 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c836c-59e8-4902-8836-1e073d07e5d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:41.149843 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c836c-59e8-4902-8836-1e073d07e5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:41.155427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c836c-59e8-4902-8836-1e073d07e5d2 admin admin] PATH_INFO: `/v3/projects/9a40cec64c314b4a90d5ed15003185c8/users/d4ce4a6f117b418994844191999f7c42/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:41.160054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c836c-59e8-4902-8836-1e073d07e5d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a40cec64c314b4a90d5ed15003185c8, user_id=d4ce4a6f117b418994844191999f7c42, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:41.162837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c836c-59e8-4902-8836-1e073d07e5d2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:41.192893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 713/1427] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:40 2022] PUT /identity/v3/projects/9a40cec64c314b4a90d5ed15003185c8/users/d4ce4a6f117b418994844191999f7c42/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:41.200380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfffbe-bbf5-49a8-8dc2-a38d3c2960a8 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:41.201212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfffbe-bbf5-49a8-8dc2-a38d3c2960a8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:41.201638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfffbe-bbf5-49a8-8dc2-a38d3c2960a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:41.257028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 715/1428] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:41.274955 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f4bcc-0305-4b94-a670-efb8ad4b1f54 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:41.278600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-de9f4bcc-0305-4b94-a670-efb8ad4b1f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:41.323305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-eadfffbe-bbf5-49a8-8dc2-a38d3c2960a8 None None] MFA Rules not processed for user `d4ce4a6f117b418994844191999f7c42`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:41.324610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-eadfffbe-bbf5-49a8-8dc2-a38d3c2960a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:41.533262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f4bcc-0305-4b94-a670-efb8ad4b1f54 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:41.629027 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 714/1429] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:40:41.643020 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfffbe-bbf5-49a8-8dc2-a38d3c2960a8 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:41.644483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-eadfffbe-bbf5-49a8-8dc2-a38d3c2960a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:41.657205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:41.767461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c75b6-b323-4a34-9414-95da448329ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:41.768704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c75b6-b323-4a34-9414-95da448329ba service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:41.768704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c75b6-b323-4a34-9414-95da448329ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:41.768704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c75b6-b323-4a34-9414-95da448329ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:41.794498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfffbe-bbf5-49a8-8dc2-a38d3c2960a8 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:41.799275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7291f4-9d0e-4058-9e96-27cf7cbae248 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:41.800212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7291f4-9d0e-4058-9e96-27cf7cbae248 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:41.800703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7291f4-9d0e-4058-9e96-27cf7cbae248 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:41.801248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7291f4-9d0e-4058-9e96-27cf7cbae248 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:41.853679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7291f4-9d0e-4058-9e96-27cf7cbae248 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:41.855807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7291f4-9d0e-4058-9e96-27cf7cbae248 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:41.880818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 715/1430] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:40:41.921283 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c75b6-b323-4a34-9414-95da448329ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:41.922831 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c75b6-b323-4a34-9414-95da448329ba service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:41.934974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 716/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:42.186008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7291f4-9d0e-4058-9e96-27cf7cbae248 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:42.209642 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7291f4-9d0e-4058-9e96-27cf7cbae248 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:42.222357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4972a61-0f45-46e0-8a95-eb20152d2514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:42.222997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4972a61-0f45-46e0-8a95-eb20152d2514 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:42.223272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4972a61-0f45-46e0-8a95-eb20152d2514 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:42.223962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4972a61-0f45-46e0-8a95-eb20152d2514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:42.413291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4972a61-0f45-46e0-8a95-eb20152d2514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:42.416730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4972a61-0f45-46e0-8a95-eb20152d2514 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:42.437471 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 716/1432] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:43.357533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c75b6-b323-4a34-9414-95da448329ba service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:43.403726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:43.522172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c75b6-b323-4a34-9414-95da448329ba service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:43.537327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b52af-7404-4b73-8626-61bcaa6b0fd9 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:43.538799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b52af-7404-4b73-8626-61bcaa6b0fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:43.540326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b52af-7404-4b73-8626-61bcaa6b0fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:43.546457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:43.549471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:43.549771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:43.553104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] PATH_INFO: `/v3/users/4c1a459b89de421889d41a0c3bdb037e` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:43.559620 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c1a459b89de421889d41a0c3bdb037e)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:43.567571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:43.597682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-724b52af-7404-4b73-8626-61bcaa6b0fd9 None None] MFA Rules not processed for user `916699a212eb4d8aa945f299ac0d4c96`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:43.602516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-724b52af-7404-4b73-8626-61bcaa6b0fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:43.696121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c1a459b89de421889d41a0c3bdb037e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:43.740554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c1a459b89de421889d41a0c3bdb037e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:43.741335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c1a459b89de421889d41a0c3bdb037e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:43.758255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c1a459b89de421889d41a0c3bdb037e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:43.759652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c1a459b89de421889d41a0c3bdb037e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:43.790195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 717/1433] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:43 2022] DELETE /identity/v3/users/4c1a459b89de421889d41a0c3bdb037e => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:43.792183 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:43.795822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:43.797649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 717/1434] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:43.800141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333ece-d752-4a58-877a-c64cfc422284 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:43.800548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333ece-d752-4a58-877a-c64cfc422284 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:43.800724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333ece-d752-4a58-877a-c64cfc422284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:43.863503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ee333ece-d752-4a58-877a-c64cfc422284 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:43.866141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ee333ece-d752-4a58-877a-c64cfc422284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:43.984663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c13c7-a739-4c34-a37e-ce6109a9b51c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:43.995388 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:43.997322 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:43.998769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:43.999051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:43.999342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] PATH_INFO: `/v3/users/825d9df94a18454d99304eea083b03a8` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:44.005575 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=825d9df94a18454d99304eea083b03a8)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:44.005575 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:44.034889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 718/1435] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:44.044898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee333ece-d752-4a58-877a-c64cfc422284 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:44.046306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ee333ece-d752-4a58-877a-c64cfc422284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:44.112319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:44.128712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '825d9df94a18454d99304eea083b03a8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:44.163393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '825d9df94a18454d99304eea083b03a8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:44.166374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '825d9df94a18454d99304eea083b03a8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:44.188425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '825d9df94a18454d99304eea083b03a8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:44.190251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a722dec-3d78-42b4-a8f2-f5d5a16dac88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '825d9df94a18454d99304eea083b03a8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:44.221043 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 718/1436] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:43 2022] DELETE /identity/v3/users/825d9df94a18454d99304eea083b03a8 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:44.224957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee333ece-d752-4a58-877a-c64cfc422284 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:44.227694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec1861-3a4f-4e91-bec9-7338bdfcb9fc None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:44.228052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec1861-3a4f-4e91-bec9-7338bdfcb9fc None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:44.228907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec1861-3a4f-4e91-bec9-7338bdfcb9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:44.261324 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-48ec1861-3a4f-4e91-bec9-7338bdfcb9fc None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:44.263356 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-48ec1861-3a4f-4e91-bec9-7338bdfcb9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:44.397277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 719/1437] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:44.398187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec1861-3a4f-4e91-bec9-7338bdfcb9fc None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:44.402287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-48ec1861-3a4f-4e91-bec9-7338bdfcb9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:44.410457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92d75f-7a76-43ba-8372-2c0edc9a99c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:44.411228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92d75f-7a76-43ba-8372-2c0edc9a99c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:44.411615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92d75f-7a76-43ba-8372-2c0edc9a99c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:44.411992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92d75f-7a76-43ba-8372-2c0edc9a99c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:44.419256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec1861-3a4f-4e91-bec9-7338bdfcb9fc None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:44.420741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4b92d75f-7a76-43ba-8372-2c0edc9a99c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:44.426211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:44.427057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:44.427503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:44.427917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] PATH_INFO: `/v3/users/b04ad85928b24b969e923ca20b2a453a` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:44.429334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04ad85928b24b969e923ca20b2a453a)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:44.433131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:44.470960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92d75f-7a76-43ba-8372-2c0edc9a99c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:44.472498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92d75f-7a76-43ba-8372-2c0edc9a99c5 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:44.498080 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 719/1438] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:40:44.499110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92d75f-7a76-43ba-8372-2c0edc9a99c5 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:44.508391 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4b92d75f-7a76-43ba-8372-2c0edc9a99c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:44.536509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92d75f-7a76-43ba-8372-2c0edc9a99c5 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:44.571581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804ad73-609e-477a-aa6d-6fb1183ea0e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:44.572708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804ad73-609e-477a-aa6d-6fb1183ea0e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:44.572925 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804ad73-609e-477a-aa6d-6fb1183ea0e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:44.573301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804ad73-609e-477a-aa6d-6fb1183ea0e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:44.573765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04ad85928b24b969e923ca20b2a453a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:44.616999 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04ad85928b24b969e923ca20b2a453a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:44.618406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04ad85928b24b969e923ca20b2a453a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:44.625771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804ad73-609e-477a-aa6d-6fb1183ea0e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:44.627526 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804ad73-609e-477a-aa6d-6fb1183ea0e8 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:44.637509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04ad85928b24b969e923ca20b2a453a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:44.638353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7654d055-db94-4a6c-bd8a-6f207d3c076d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04ad85928b24b969e923ca20b2a453a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:44.672790 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 720/1439] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:44 2022] DELETE /identity/v3/users/b04ad85928b24b969e923ca20b2a453a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:44.674980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 720/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:44.678614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71426564-0775-4930-85fd-cce7e1b10649 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:44.678907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71426564-0775-4930-85fd-cce7e1b10649 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:44.679188 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71426564-0775-4930-85fd-cce7e1b10649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:44.689599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4abc-ebe0-4fc9-8e6a-c6d1feda90cd None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:44.690094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4abc-ebe0-4fc9-8e6a-c6d1feda90cd None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:44.690544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4abc-ebe0-4fc9-8e6a-c6d1feda90cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:44.717671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-71426564-0775-4930-85fd-cce7e1b10649 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:44.721782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-71426564-0775-4930-85fd-cce7e1b10649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:44.795444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:44.811292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ab0a4abc-ebe0-4fc9-8e6a-c6d1feda90cd None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:44.812527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ab0a4abc-ebe0-4fc9-8e6a-c6d1feda90cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:44.887297 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 721/1441] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:44.973529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 721/1442] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:40:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:40:44.984209 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71426564-0775-4930-85fd-cce7e1b10649 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:44.987566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-71426564-0775-4930-85fd-cce7e1b10649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:45.114116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71426564-0775-4930-85fd-cce7e1b10649 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:45.119943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e50b2-353e-4c62-a14f-8d9c2a2eeb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:45.120578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e50b2-353e-4c62-a14f-8d9c2a2eeb46 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:45.121154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e50b2-353e-4c62-a14f-8d9c2a2eeb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:45.122105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e50b2-353e-4c62-a14f-8d9c2a2eeb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:45.138401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e50b2-353e-4c62-a14f-8d9c2a2eeb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:45.140323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e50b2-353e-4c62-a14f-8d9c2a2eeb46 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:45.161740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 722/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:45.303432 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a4abc-ebe0-4fc9-8e6a-c6d1feda90cd None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:45.305558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ab0a4abc-ebe0-4fc9-8e6a-c6d1feda90cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:45.330875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:45.619392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a4abc-ebe0-4fc9-8e6a-c6d1feda90cd None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:45.626191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf858a-11ad-47ce-bd1c-db2c02065e2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:45.626975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf858a-11ad-47ce-bd1c-db2c02065e2f service swift] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:45.627081 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf858a-11ad-47ce-bd1c-db2c02065e2f service swift] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:45.627326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf858a-11ad-47ce-bd1c-db2c02065e2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:45.639769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf858a-11ad-47ce-bd1c-db2c02065e2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:45.641689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf858a-11ad-47ce-bd1c-db2c02065e2f service swift] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:45.660430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 722/1444] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:40:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:40:46.486538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e50b2-353e-4c62-a14f-8d9c2a2eeb46 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:46.487992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d15e50b2-353e-4c62-a14f-8d9c2a2eeb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:46.517848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e50b2-353e-4c62-a14f-8d9c2a2eeb46 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:46.547259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:46.547932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:46.548210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:46.548483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] PATH_INFO: `/v3/users/da571ebd580349c1bf9af12bf8ec4fdf` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:46.555486 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da571ebd580349c1bf9af12bf8ec4fdf)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:46.558853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:46.591004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf858a-11ad-47ce-bd1c-db2c02065e2f service swift] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:46.597783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-89bf858a-11ad-47ce-bd1c-db2c02065e2f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:46.658336 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf858a-11ad-47ce-bd1c-db2c02065e2f service swift] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:46.679929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:46.680584 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:46.680883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:46.681161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] PATH_INFO: `/v3/projects/862d049d044e442391fbd4bf36db3414` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:46.691628 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=862d049d044e442391fbd4bf36db3414)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:46.693769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:46.726423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da571ebd580349c1bf9af12bf8ec4fdf'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:46.772491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da571ebd580349c1bf9af12bf8ec4fdf'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:46.774066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da571ebd580349c1bf9af12bf8ec4fdf'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:46.817746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da571ebd580349c1bf9af12bf8ec4fdf'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:46.820566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67586603-4c2f-4b57-808b-739279db62a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da571ebd580349c1bf9af12bf8ec4fdf'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:46.897129 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 723/1445] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:46 2022] DELETE /identity/v3/users/da571ebd580349c1bf9af12bf8ec4fdf => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:46.905624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221e5fc-e11b-456b-9d8a-0fcc2527bd18 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:46.905923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221e5fc-e11b-456b-9d8a-0fcc2527bd18 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:46.906216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221e5fc-e11b-456b-9d8a-0fcc2527bd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:46.947252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862d049d044e442391fbd4bf36db3414', 'request_id': 'req-556948c0-6967-4fb2-ae46-b37aa1e72343'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:46.973222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862d049d044e442391fbd4bf36db3414', 'request_id': 'req-556948c0-6967-4fb2-ae46-b37aa1e72343'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:46.990563 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d221e5fc-e11b-456b-9d8a-0fcc2527bd18 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:46.992133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d221e5fc-e11b-456b-9d8a-0fcc2527bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:47.009730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862d049d044e442391fbd4bf36db3414', 'request_id': 'req-556948c0-6967-4fb2-ae46-b37aa1e72343'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:47.078361 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] The token cache is being invalidate because project 862d049d044e442391fbd4bf36db3414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:47.080014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:47.082026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:47.088076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 723/1446] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:46 2022] DELETE /identity/v3/projects/862d049d044e442391fbd4bf36db3414 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:47.097278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:47.105927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:47.115508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:47.165073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 724/1447] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:47.338736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221e5fc-e11b-456b-9d8a-0fcc2527bd18 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:47.341015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d221e5fc-e11b-456b-9d8a-0fcc2527bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:47.383997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556948c0-6967-4fb2-ae46-b37aa1e72343 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:47.405589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:47.405589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:47.405589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:47.405589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] PATH_INFO: `/v3/users/978f4c97d5a74171a3466e6b33e032a8` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:47.405989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=978f4c97d5a74171a3466e6b33e032a8)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:47.408705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:47.415180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:47.553894 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '978f4c97d5a74171a3466e6b33e032a8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:47.583301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '978f4c97d5a74171a3466e6b33e032a8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:47.584525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '978f4c97d5a74171a3466e6b33e032a8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:47.633691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221e5fc-e11b-456b-9d8a-0fcc2527bd18 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:47.637546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '978f4c97d5a74171a3466e6b33e032a8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:47.640623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c836ec6-2956-4f0b-ae10-48a7ceaa258f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:47.641727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c836ec6-2956-4f0b-ae10-48a7ceaa258f service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:47.642763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c836ec6-2956-4f0b-ae10-48a7ceaa258f service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:47.643201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c836ec6-2956-4f0b-ae10-48a7ceaa258f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:47.645543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0c836ec6-2956-4f0b-ae10-48a7ceaa258f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:47.671811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c836ec6-2956-4f0b-ae10-48a7ceaa258f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:47.673845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c836ec6-2956-4f0b-ae10-48a7ceaa258f service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:47.674002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '978f4c97d5a74171a3466e6b33e032a8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:47.711328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 724/1448] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:47 2022] DELETE /identity/v3/users/978f4c97d5a74171a3466e6b33e032a8 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:47.722681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 725/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:40:47.771382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:47.773152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:47.988164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32510397-a65d-4219-9b93-fa792b846e0a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:48.010619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:48.021865 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:48.022522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:48.022966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] PATH_INFO: `/v3/projects/60dc23e43cc34c39a77700b1d329ce81` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:48.024476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60dc23e43cc34c39a77700b1d329ce81)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:48.026618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:48.194648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60dc23e43cc34c39a77700b1d329ce81', 'request_id': 'req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:48.240320 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60dc23e43cc34c39a77700b1d329ce81', 'request_id': 'req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:48.297501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60dc23e43cc34c39a77700b1d329ce81', 'request_id': 'req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:48.319740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] The token cache is being invalidate because project 60dc23e43cc34c39a77700b1d329ce81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:48.320250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:48.321836 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:48.324934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 725/1450] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:47 2022] DELETE /identity/v3/projects/60dc23e43cc34c39a77700b1d329ce81 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:48.329146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c836ec6-2956-4f0b-ae10-48a7ceaa258f service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:48.357271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c836ec6-2956-4f0b-ae10-48a7ceaa258f service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:48.594737 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:48.599891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:48.600650 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:48.601603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] PATH_INFO: `/v3/users/d4ce4a6f117b418994844191999f7c42` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:48.603774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ce4a6f117b418994844191999f7c42)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:48.606719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:48.736777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ce4a6f117b418994844191999f7c42'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:48.771796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ce4a6f117b418994844191999f7c42'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:48.773892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ce4a6f117b418994844191999f7c42'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:48.810883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ce4a6f117b418994844191999f7c42'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:48.813506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ce4a6f117b418994844191999f7c42'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:48.856555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 726/1451] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:48 2022] DELETE /identity/v3/users/d4ce4a6f117b418994844191999f7c42 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:49.770954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:49.774186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:49.806706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d577d82-e2c0-4eb3-8573-41be4e0c2fd5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:49.877776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:49.930820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:49.932546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:49.989862 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:49.990773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:49.991204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:49.991635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] PATH_INFO: `/v3/projects/a49aaaef7c9140539fb2704baf87c159` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:49.992998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49aaaef7c9140539fb2704baf87c159)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:49.995658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:49.998615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:50.101958 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49aaaef7c9140539fb2704baf87c159', 'request_id': 'req-b1e1d274-c846-4211-8a86-f3ad48190a52'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.112720 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49aaaef7c9140539fb2704baf87c159', 'request_id': 'req-b1e1d274-c846-4211-8a86-f3ad48190a52'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.124406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49aaaef7c9140539fb2704baf87c159', 'request_id': 'req-b1e1d274-c846-4211-8a86-f3ad48190a52'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.177152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5a9fc-a683-4d7d-a935-a4b95ff1038c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:50.177611 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] The token cache is being invalidate because project a49aaaef7c9140539fb2704baf87c159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:50.177914 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.182691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b1e1d274-c846-4211-8a86-f3ad48190a52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.192076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 726/1452] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:49 2022] DELETE /identity/v3/projects/a49aaaef7c9140539fb2704baf87c159 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:50.196717 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130f4c5-aac1-46b4-9d5f-532881104cd3 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:50.198128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130f4c5-aac1-46b4-9d5f-532881104cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:50.201148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130f4c5-aac1-46b4-9d5f-532881104cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:50.305222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-c130f4c5-aac1-46b4-9d5f-532881104cd3 None None] MFA Rules not processed for user `d301f3d9aeb644a38c31dbdbef2a48c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:50.306484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c130f4c5-aac1-46b4-9d5f-532881104cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:50.345301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:50.349256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:50.349704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:50.350123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] PATH_INFO: `/v3/projects/950c2a3d60ac432cb8a76d53df60b0a4` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:50.353029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] RBAC: Authorizing `identity:delete_project(project_id=950c2a3d60ac432cb8a76d53df60b0a4)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:50.360442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:50.468801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 727/1453] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Oct 18 11:40:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:50.470681 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130f4c5-aac1-46b4-9d5f-532881104cd3 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:50.474890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c130f4c5-aac1-46b4-9d5f-532881104cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:50.507104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950c2a3d60ac432cb8a76d53df60b0a4', 'request_id': 'req-f052b910-b5be-4840-b42e-eabf3ac7f267'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.522723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130f4c5-aac1-46b4-9d5f-532881104cd3 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:50.524695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950c2a3d60ac432cb8a76d53df60b0a4', 'request_id': 'req-f052b910-b5be-4840-b42e-eabf3ac7f267'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.543997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950c2a3d60ac432cb8a76d53df60b0a4', 'request_id': 'req-f052b910-b5be-4840-b42e-eabf3ac7f267'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.575892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] The token cache is being invalidate because project 950c2a3d60ac432cb8a76d53df60b0a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:50.576334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.578261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f052b910-b5be-4840-b42e-eabf3ac7f267 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.580884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 727/1454] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:49 2022] DELETE /identity/v3/projects/950c2a3d60ac432cb8a76d53df60b0a4 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:40:50.595513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73275d6-e352-4ac8-bccb-672c9bee2211 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:50.599024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73275d6-e352-4ac8-bccb-672c9bee2211 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:50.604561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73275d6-e352-4ac8-bccb-672c9bee2211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:50.685320 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e73275d6-e352-4ac8-bccb-672c9bee2211 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:50.686552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e73275d6-e352-4ac8-bccb-672c9bee2211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:50.705533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:50.791095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:50.794481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:50.796623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:50.796623 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] PATH_INFO: `/v3/users/dd3b214588aa49c89f4d91c93546cf0d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:50.802056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd3b214588aa49c89f4d91c93546cf0d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:50.804061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:50.923825 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:50.953451 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd3b214588aa49c89f4d91c93546cf0d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.978205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd3b214588aa49c89f4d91c93546cf0d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:50.981560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd3b214588aa49c89f4d91c93546cf0d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:51.011199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 728/1455] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:51.012173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73275d6-e352-4ac8-bccb-672c9bee2211 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:51.016642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e73275d6-e352-4ac8-bccb-672c9bee2211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:51.020113 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd3b214588aa49c89f4d91c93546cf0d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:51.042005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd3b214588aa49c89f4d91c93546cf0d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:51.056391 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 728/1456] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:50 2022] DELETE /identity/v3/users/dd3b214588aa49c89f4d91c93546cf0d => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:51.057393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:51.060028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:51.178219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57b78a-7d19-4d75-ab2f-a41d55c55a55 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:51.195111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfc3bf-72d0-4df7-add7-e2b6dd9d69d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:51.195774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfc3bf-72d0-4df7-add7-e2b6dd9d69d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:51.196111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfc3bf-72d0-4df7-add7-e2b6dd9d69d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:51.196371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfc3bf-72d0-4df7-add7-e2b6dd9d69d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:51.198339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cdbfc3bf-72d0-4df7-add7-e2b6dd9d69d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:51.203832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73275d6-e352-4ac8-bccb-672c9bee2211 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:51.218988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfc3bf-72d0-4df7-add7-e2b6dd9d69d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:51.220444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfc3bf-72d0-4df7-add7-e2b6dd9d69d7 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:51.220787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00c0ff-5cdb-45cc-a423-636191d5a982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:51.222824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00c0ff-5cdb-45cc-a423-636191d5a982 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:51.224450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00c0ff-5cdb-45cc-a423-636191d5a982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:51.224774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00c0ff-5cdb-45cc-a423-636191d5a982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:51.242379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 729/1457] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:51.244978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9827e49-ff43-4753-a551-9ffa48bc9a1b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:51.245288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9827e49-ff43-4753-a551-9ffa48bc9a1b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:51.245672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9827e49-ff43-4753-a551-9ffa48bc9a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:51.263711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00c0ff-5cdb-45cc-a423-636191d5a982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:51.272638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b9827e49-ff43-4753-a551-9ffa48bc9a1b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:51.274319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b9827e49-ff43-4753-a551-9ffa48bc9a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:51.276538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00c0ff-5cdb-45cc-a423-636191d5a982 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:51.308154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 729/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:51.372620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 730/1459] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:51.374028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00c0ff-5cdb-45cc-a423-636191d5a982 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:51.402169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00c0ff-5cdb-45cc-a423-636191d5a982 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:51.426316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082c4d2-ce30-4d0b-8021-f9baab1bea84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:51.429385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c4d2-ce30-4d0b-8021-f9baab1bea84 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:51.429924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c4d2-ce30-4d0b-8021-f9baab1bea84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:51.430859 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c4d2-ce30-4d0b-8021-f9baab1bea84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:51.432571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082c4d2-ce30-4d0b-8021-f9baab1bea84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:51.434418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082c4d2-ce30-4d0b-8021-f9baab1bea84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:51.439340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082c4d2-ce30-4d0b-8021-f9baab1bea84 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:51.477115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 730/1460] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:40:51.478792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9827e49-ff43-4753-a551-9ffa48bc9a1b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:51.491510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9827e49-ff43-4753-a551-9ffa48bc9a1b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:51.505918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a8f2e-27b9-4ada-b3ac-6f3311d25138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:51.506125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a8f2e-27b9-4ada-b3ac-6f3311d25138 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:51.506463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a8f2e-27b9-4ada-b3ac-6f3311d25138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:51.506744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a8f2e-27b9-4ada-b3ac-6f3311d25138 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:51.507728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a8f2e-27b9-4ada-b3ac-6f3311d25138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:51.510447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a8f2e-27b9-4ada-b3ac-6f3311d25138 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:51.524715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:51.568293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 731/1461] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:51 2022] POST /identity/v3/projects => generated 387 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:40:51.571688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082c4d2-ce30-4d0b-8021-f9baab1bea84 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:51.596377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082c4d2-ce30-4d0b-8021-f9baab1bea84 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:51.783992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a8f2e-27b9-4ada-b3ac-6f3311d25138 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:51.786130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a24a8f2e-27b9-4ada-b3ac-6f3311d25138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:51.842286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fc009-13cd-47af-8c72-1f7ef4a0f7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:51.848473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fc009-13cd-47af-8c72-1f7ef4a0f7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:51.849234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fc009-13cd-47af-8c72-1f7ef4a0f7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:51.852196 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fc009-13cd-47af-8c72-1f7ef4a0f7d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:51.853730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fc009-13cd-47af-8c72-1f7ef4a0f7d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:51.866499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fc009-13cd-47af-8c72-1f7ef4a0f7d3 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:51.936828 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 731/1462] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:51 2022] POST /identity/v3/users => generated 294 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:51.940683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fc009-13cd-47af-8c72-1f7ef4a0f7d3 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:51.977256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fc009-13cd-47af-8c72-1f7ef4a0f7d3 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:51.995212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56f278-c16d-4ad5-ab56-7a1c88d5883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:51.997181 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56f278-c16d-4ad5-ab56-7a1c88d5883b admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:52.000936 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56f278-c16d-4ad5-ab56-7a1c88d5883b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:52.001228 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56f278-c16d-4ad5-ab56-7a1c88d5883b admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:52.002197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56f278-c16d-4ad5-ab56-7a1c88d5883b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:52.004009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56f278-c16d-4ad5-ab56-7a1c88d5883b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:52.004889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a8f2e-27b9-4ada-b3ac-6f3311d25138 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:52.021232 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:52.022133 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:52.022875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:52.023358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] PATH_INFO: `/v3/projects/15fafabc29464e789bc4fa83f3389a20` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:52.025954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] RBAC: Authorizing `identity:delete_project(project_id=15fafabc29464e789bc4fa83f3389a20)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:52.028357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:52.038196 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 732/1463] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:40:52.039853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56f278-c16d-4ad5-ab56-7a1c88d5883b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:52.070923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56f278-c16d-4ad5-ab56-7a1c88d5883b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:52.098817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80ab7b-abb6-4a18-a6cd-735c8d9c83bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:52.099761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80ab7b-abb6-4a18-a6cd-735c8d9c83bf admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:52.100051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80ab7b-abb6-4a18-a6cd-735c8d9c83bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:52.100324 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80ab7b-abb6-4a18-a6cd-735c8d9c83bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:52.101076 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80ab7b-abb6-4a18-a6cd-735c8d9c83bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:52.104889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80ab7b-abb6-4a18-a6cd-735c8d9c83bf admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:52.143899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 733/1464] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:52.147771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80ab7b-abb6-4a18-a6cd-735c8d9c83bf admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:52.184516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80ab7b-abb6-4a18-a6cd-735c8d9c83bf admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:52.185291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15fafabc29464e789bc4fa83f3389a20', 'request_id': 'req-930f5c71-c273-4adc-8171-d141d533590c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.197189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15fafabc29464e789bc4fa83f3389a20', 'request_id': 'req-930f5c71-c273-4adc-8171-d141d533590c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.220507 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15fafabc29464e789bc4fa83f3389a20', 'request_id': 'req-930f5c71-c273-4adc-8171-d141d533590c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.246843 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] The token cache is being invalidate because project 15fafabc29464e789bc4fa83f3389a20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:52.247380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.249019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.265949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 732/1465] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:51 2022] DELETE /identity/v3/projects/15fafabc29464e789bc4fa83f3389a20 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:52.271191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:52.275672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:52.295818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:52.344722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c2c33-df27-45dd-9dfe-1e4f82516e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:52.345493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c2c33-df27-45dd-9dfe-1e4f82516e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:52.346671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c2c33-df27-45dd-9dfe-1e4f82516e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:52.346671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c2c33-df27-45dd-9dfe-1e4f82516e1c admin admin] PATH_INFO: `/v3/projects/2a43dd88e77c4a66b30a2e63e5e7103a/users/8a8733f56d6d4a65b1af34a9b44c0ce9/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:52.396016 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c2c33-df27-45dd-9dfe-1e4f82516e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a43dd88e77c4a66b30a2e63e5e7103a, user_id=8a8733f56d6d4a65b1af34a9b44c0ce9, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:52.399570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c2c33-df27-45dd-9dfe-1e4f82516e1c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:52.451551 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 734/1466] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:52 2022] PUT /identity/v3/projects/2a43dd88e77c4a66b30a2e63e5e7103a/users/8a8733f56d6d4a65b1af34a9b44c0ce9/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:52.453554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83907ca0-b5a1-42ae-aef9-d7f546852753 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:52.453839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83907ca0-b5a1-42ae-aef9-d7f546852753 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:52.454119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83907ca0-b5a1-42ae-aef9-d7f546852753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:52.492009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f5c71-c273-4adc-8171-d141d533590c admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:52.535557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-83907ca0-b5a1-42ae-aef9-d7f546852753 None None] MFA Rules not processed for user `8a8733f56d6d4a65b1af34a9b44c0ce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:52.536715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-83907ca0-b5a1-42ae-aef9-d7f546852753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:52.565750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:52.669016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:52.669691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:52.669971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:52.670244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] PATH_INFO: `/v3/users/a540ae80623344d3bf7ec6517357d9f5` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:52.677810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a540ae80623344d3bf7ec6517357d9f5)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:52.679804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:52.757233 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 735/1467] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:52.758300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83907ca0-b5a1-42ae-aef9-d7f546852753 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:52.762398 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-83907ca0-b5a1-42ae-aef9-d7f546852753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:52.786555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a540ae80623344d3bf7ec6517357d9f5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.787317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83907ca0-b5a1-42ae-aef9-d7f546852753 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:52.794056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:52.794493 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:52.794774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:52.795044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] PATH_INFO: `/v3/projects/9a40cec64c314b4a90d5ed15003185c8` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:52.796614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a40cec64c314b4a90d5ed15003185c8)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:52.799527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:52.803142 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a540ae80623344d3bf7ec6517357d9f5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.811346 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a540ae80623344d3bf7ec6517357d9f5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.844510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a540ae80623344d3bf7ec6517357d9f5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.856601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a540ae80623344d3bf7ec6517357d9f5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.867536 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 733/1468] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:40:52 2022] DELETE /identity/v3/users/a540ae80623344d3bf7ec6517357d9f5 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:40:52.869745 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:52.872936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:52.948452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a40cec64c314b4a90d5ed15003185c8', 'request_id': 'req-e4ce6f76-d84b-419d-8e80-be1a9785c246'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.964256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a40cec64c314b4a90d5ed15003185c8', 'request_id': 'req-e4ce6f76-d84b-419d-8e80-be1a9785c246'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.983675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a40cec64c314b4a90d5ed15003185c8', 'request_id': 'req-e4ce6f76-d84b-419d-8e80-be1a9785c246'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:52.990271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f649f-13c4-4e10-9cba-eb891dedbc85 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:53.015059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] The token cache is being invalidate because project 9a40cec64c314b4a90d5ed15003185c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:53.021615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:53.022671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e4ce6f76-d84b-419d-8e80-be1a9785c246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:53.029077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 736/1469] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:52 2022] DELETE /identity/v3/projects/9a40cec64c314b4a90d5ed15003185c8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:53.030620 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693e3ee-08b0-4808-8819-bd91dc9437ad None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:53.030909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693e3ee-08b0-4808-8819-bd91dc9437ad None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:53.031177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693e3ee-08b0-4808-8819-bd91dc9437ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:53.079015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6693e3ee-08b0-4808-8819-bd91dc9437ad None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:53.080631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6693e3ee-08b0-4808-8819-bd91dc9437ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:53.132716 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:53.149961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3c7e9-f918-4fea-af20-200c3f925aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:53.150571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3c7e9-f918-4fea-af20-200c3f925aea admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:53.150850 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3c7e9-f918-4fea-af20-200c3f925aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:53.151314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3c7e9-f918-4fea-af20-200c3f925aea admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:53.152689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3c7e9-f918-4fea-af20-200c3f925aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:53.154679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3c7e9-f918-4fea-af20-200c3f925aea admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:53.154825 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:53.216768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 734/1470] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:52 2022] POST /identity/v3/projects => generated 385 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:53.219177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3c7e9-f918-4fea-af20-200c3f925aea admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:53.221779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-efb3c7e9-f918-4fea-af20-200c3f925aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:53.242089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 737/1471] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:53.244946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c745d7-c56f-48ab-82e4-cfc42844b279 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:53.245342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c745d7-c56f-48ab-82e4-cfc42844b279 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:53.245609 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c745d7-c56f-48ab-82e4-cfc42844b279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:53.279425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-93c745d7-c56f-48ab-82e4-cfc42844b279 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:53.281503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-93c745d7-c56f-48ab-82e4-cfc42844b279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:53.382261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 738/1472] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:53.383571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c745d7-c56f-48ab-82e4-cfc42844b279 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:53.407839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c745d7-c56f-48ab-82e4-cfc42844b279 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:53.426808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93171e62-36b8-4838-b201-fd10d9fd7f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:53.427547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93171e62-36b8-4838-b201-fd10d9fd7f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:53.427831 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93171e62-36b8-4838-b201-fd10d9fd7f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:53.428104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93171e62-36b8-4838-b201-fd10d9fd7f25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:53.428773 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93171e62-36b8-4838-b201-fd10d9fd7f25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:53.432639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93171e62-36b8-4838-b201-fd10d9fd7f25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:53.440736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93171e62-36b8-4838-b201-fd10d9fd7f25 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:53.484005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3c7e9-f918-4fea-af20-200c3f925aea admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:53.509515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 739/1473] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:53.509706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76679bfc-e9ed-4d41-9241-91fe69656c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:53.509798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93171e62-36b8-4838-b201-fd10d9fd7f25 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:53.509867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76679bfc-e9ed-4d41-9241-91fe69656c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:53.509867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76679bfc-e9ed-4d41-9241-91fe69656c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:53.509867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76679bfc-e9ed-4d41-9241-91fe69656c8a admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:53.509867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76679bfc-e9ed-4d41-9241-91fe69656c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:53.510751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76679bfc-e9ed-4d41-9241-91fe69656c8a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:53.514595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-93171e62-36b8-4838-b201-fd10d9fd7f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:53.555033 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93171e62-36b8-4838-b201-fd10d9fd7f25 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:53.578477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 735/1474] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:53 2022] POST /identity/v3/users => generated 293 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:53.580836 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76679bfc-e9ed-4d41-9241-91fe69656c8a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:53.606387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76679bfc-e9ed-4d41-9241-91fe69656c8a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:53.619059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4b061-0760-4cb0-98ec-b98065e6f38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:53.619734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4b061-0760-4cb0-98ec-b98065e6f38d admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:53.619969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4b061-0760-4cb0-98ec-b98065e6f38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:53.620228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4b061-0760-4cb0-98ec-b98065e6f38d admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:53.621490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4b061-0760-4cb0-98ec-b98065e6f38d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:53.626747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4b061-0760-4cb0-98ec-b98065e6f38d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:53.628658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:53.629379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:53.629722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:53.630060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] PATH_INFO: `/v3/projects/e3990d114f5540bdaf141918421f1ad8` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:53.632460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3990d114f5540bdaf141918421f1ad8)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:53.638276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:53.774612 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 736/1475] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:53 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:53.783355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4b061-0760-4cb0-98ec-b98065e6f38d admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:53.812751 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3990d114f5540bdaf141918421f1ad8', 'request_id': 'req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:53.836941 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3990d114f5540bdaf141918421f1ad8', 'request_id': 'req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:53.864794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3990d114f5540bdaf141918421f1ad8', 'request_id': 'req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:53.894991 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] The token cache is being invalidate because project e3990d114f5540bdaf141918421f1ad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:53.895340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:53.896072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:53.897960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 740/1476] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:53 2022] DELETE /identity/v3/projects/e3990d114f5540bdaf141918421f1ad8 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:53.898807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:53.900552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:54.014727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4b061-0760-4cb0-98ec-b98065e6f38d admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:54.099209 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:54.133896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7043e48-cc4f-49e3-8ccc-a2fe5c3c5580 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:54.139604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38e0b5-2db8-4879-8451-3c5ccb375531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:54.140289 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38e0b5-2db8-4879-8451-3c5ccb375531 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.140580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38e0b5-2db8-4879-8451-3c5ccb375531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.140888 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38e0b5-2db8-4879-8451-3c5ccb375531 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.141540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38e0b5-2db8-4879-8451-3c5ccb375531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:54.143531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38e0b5-2db8-4879-8451-3c5ccb375531 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:54.197187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 741/1477] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:53 2022] POST /identity/v3/users => generated 287 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:54.199465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc0f20-930b-4ac4-819a-fc62477a5217 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.199947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc0f20-930b-4ac4-819a-fc62477a5217 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.200410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc0f20-930b-4ac4-819a-fc62477a5217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.229044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6ecc0f20-930b-4ac4-819a-fc62477a5217 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:54.230516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6ecc0f20-930b-4ac4-819a-fc62477a5217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:54.256164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:54.267420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 742/1478] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:54.268315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc0f20-930b-4ac4-819a-fc62477a5217 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:54.287391 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d813f-1684-4a87-91a4-38cff6b468fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:54.288166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d813f-1684-4a87-91a4-38cff6b468fd admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.288390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d813f-1684-4a87-91a4-38cff6b468fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.288662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d813f-1684-4a87-91a4-38cff6b468fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.289362 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d813f-1684-4a87-91a4-38cff6b468fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:54.290461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc0f20-930b-4ac4-819a-fc62477a5217 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:54.304875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d813f-1684-4a87-91a4-38cff6b468fd admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:54.308419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3d0c6-ca85-4e24-821c-52fbfd7aaeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:54.309132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3d0c6-ca85-4e24-821c-52fbfd7aaeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.309417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3d0c6-ca85-4e24-821c-52fbfd7aaeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.309694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3d0c6-ca85-4e24-821c-52fbfd7aaeb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.310359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3d0c6-ca85-4e24-821c-52fbfd7aaeb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:54.312184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3d0c6-ca85-4e24-821c-52fbfd7aaeb4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:54.339520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 743/1479] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:54.341998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec054f0-4d70-415f-9f5f-1124133b7880 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.342294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec054f0-4d70-415f-9f5f-1124133b7880 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.343036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec054f0-4d70-415f-9f5f-1124133b7880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.362628 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 737/1480] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 582 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:54.363874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d813f-1684-4a87-91a4-38cff6b468fd admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:54.378685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d813f-1684-4a87-91a4-38cff6b468fd admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:54.389887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef871b8-f90b-46a4-850c-35fa90df01f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:54.390818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef871b8-f90b-46a4-850c-35fa90df01f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.391590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef871b8-f90b-46a4-850c-35fa90df01f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.392083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef871b8-f90b-46a4-850c-35fa90df01f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.394342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-8ec054f0-4d70-415f-9f5f-1124133b7880 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:54.395687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8ec054f0-4d70-415f-9f5f-1124133b7880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:54.402001 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef871b8-f90b-46a4-850c-35fa90df01f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:54.405190 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef871b8-f90b-46a4-850c-35fa90df01f7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:54.418920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 744/1481] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:54.419947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec054f0-4d70-415f-9f5f-1124133b7880 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:54.421779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8ec054f0-4d70-415f-9f5f-1124133b7880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:54.447883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec054f0-4d70-415f-9f5f-1124133b7880 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:54.456271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 738/1482] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:54.460230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef871b8-f90b-46a4-850c-35fa90df01f7 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:54.463080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a4bbc-cfd1-404b-8659-8cdf3f9ef2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:54.464105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a4bbc-cfd1-404b-8659-8cdf3f9ef2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.465319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a4bbc-cfd1-404b-8659-8cdf3f9ef2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.465780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a4bbc-cfd1-404b-8659-8cdf3f9ef2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.467029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a4bbc-cfd1-404b-8659-8cdf3f9ef2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:54.472609 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a4bbc-cfd1-404b-8659-8cdf3f9ef2b9 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:54.494312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 745/1483] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:54.499350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a4bbc-cfd1-404b-8659-8cdf3f9ef2b9 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:54.506014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef871b8-f90b-46a4-850c-35fa90df01f7 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:54.524923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a4bbc-cfd1-404b-8659-8cdf3f9ef2b9 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:54.531403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c207a-36a9-49e9-bb9d-5793af19ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:54.532282 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c207a-36a9-49e9-bb9d-5793af19ca71 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.532602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dbaa4-4e20-41a7-8d72-184a1892cb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:54.532722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c207a-36a9-49e9-bb9d-5793af19ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.533070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c207a-36a9-49e9-bb9d-5793af19ca71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.533666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dbaa4-4e20-41a7-8d72-184a1892cb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.533984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c207a-36a9-49e9-bb9d-5793af19ca71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:54.534086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dbaa4-4e20-41a7-8d72-184a1892cb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.534433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c207a-36a9-49e9-bb9d-5793af19ca71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:54.534567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dbaa4-4e20-41a7-8d72-184a1892cb38 admin admin] PATH_INFO: `/v3/projects/25a8594d38034ae7805635a7b2de8ccb/users/eb41f09835414ed1a585f2485bb442cc/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.536693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c207a-36a9-49e9-bb9d-5793af19ca71 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:54.566992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 739/1484] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:54.568855 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c207a-36a9-49e9-bb9d-5793af19ca71 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:54.604963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c207a-36a9-49e9-bb9d-5793af19ca71 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:54.614231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dbaa4-4e20-41a7-8d72-184a1892cb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a8594d38034ae7805635a7b2de8ccb, user_id=eb41f09835414ed1a585f2485bb442cc, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:54.619058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fc9cb-6854-40ea-9ee5-93e8792fd731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:54.620601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fc9cb-6854-40ea-9ee5-93e8792fd731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.621715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fc9cb-6854-40ea-9ee5-93e8792fd731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.622238 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fc9cb-6854-40ea-9ee5-93e8792fd731 admin admin] PATH_INFO: `/v3/projects/216b9a0f4e3f4efaad8cb3d4b1d3be92/users/72db2349f3a7489eb4242f7a35df7a44/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.637297 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dbaa4-4e20-41a7-8d72-184a1892cb38 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:54.645691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fc9cb-6854-40ea-9ee5-93e8792fd731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=216b9a0f4e3f4efaad8cb3d4b1d3be92, user_id=72db2349f3a7489eb4242f7a35df7a44, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:54.653904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fc9cb-6854-40ea-9ee5-93e8792fd731 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:54.676907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 746/1485] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:54 2022] PUT /identity/v3/projects/25a8594d38034ae7805635a7b2de8ccb/users/eb41f09835414ed1a585f2485bb442cc/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:54.689844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dbaa4-4e20-41a7-8d72-184a1892cb38 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:54.695522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 740/1486] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:54 2022] PUT /identity/v3/projects/216b9a0f4e3f4efaad8cb3d4b1d3be92/users/72db2349f3a7489eb4242f7a35df7a44/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:54.697420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8933faa-c6f0-453d-bbe0-428541cbdb21 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.697970 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8933faa-c6f0-453d-bbe0-428541cbdb21 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.700127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8933faa-c6f0-453d-bbe0-428541cbdb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.736368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a8933faa-c6f0-453d-bbe0-428541cbdb21 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:54.738527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a8933faa-c6f0-453d-bbe0-428541cbdb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:54.852453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dbaa4-4e20-41a7-8d72-184a1892cb38 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:54.934817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 741/1487] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:54.936670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a27801-68a1-4bdb-8b3a-631c1d04e81a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:54.937270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a27801-68a1-4bdb-8b3a-631c1d04e81a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:54.937824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a27801-68a1-4bdb-8b3a-631c1d04e81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:54.978864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-97a27801-68a1-4bdb-8b3a-631c1d04e81a None None] MFA Rules not processed for user `72db2349f3a7489eb4242f7a35df7a44`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:54.981794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-97a27801-68a1-4bdb-8b3a-631c1d04e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:55.046311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b38f7c-baf9-45a4-93f6-4b5108a20246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:55.046981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b38f7c-baf9-45a4-93f6-4b5108a20246 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.047273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b38f7c-baf9-45a4-93f6-4b5108a20246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.047668 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b38f7c-baf9-45a4-93f6-4b5108a20246 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.048515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b38f7c-baf9-45a4-93f6-4b5108a20246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:55.050758 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b38f7c-baf9-45a4-93f6-4b5108a20246 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:55.098017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 742/1488] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:55.098017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a27801-68a1-4bdb-8b3a-631c1d04e81a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:55.106774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 747/1489] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:54 2022] POST /identity/v3/projects => generated 369 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:55.109252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-97a27801-68a1-4bdb-8b3a-631c1d04e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:55.113065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b38f7c-baf9-45a4-93f6-4b5108a20246 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:55.149087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:55.219400 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:55.256110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b38f7c-baf9-45a4-93f6-4b5108a20246 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:55.270559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a27801-68a1-4bdb-8b3a-631c1d04e81a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:55.287254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b4a29-90cf-4f45-b881-c2320c61d3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:55.288144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b4a29-90cf-4f45-b881-c2320c61d3ea admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.288601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b4a29-90cf-4f45-b881-c2320c61d3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.289057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b4a29-90cf-4f45-b881-c2320c61d3ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.289864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b4a29-90cf-4f45-b881-c2320c61d3ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:55.292384 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b4a29-90cf-4f45-b881-c2320c61d3ea admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:55.355802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 748/1490] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:55 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:55.360774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b4a29-90cf-4f45-b881-c2320c61d3ea admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:55.392948 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8303ee-eb34-486b-accb-24cd439cd190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:55.395600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8303ee-eb34-486b-accb-24cd439cd190 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.397439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8303ee-eb34-486b-accb-24cd439cd190 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.397722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8303ee-eb34-486b-accb-24cd439cd190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.435841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8303ee-eb34-486b-accb-24cd439cd190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:55.442049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8303ee-eb34-486b-accb-24cd439cd190 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:55.468541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 743/1491] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:55.469639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8303ee-eb34-486b-accb-24cd439cd190 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:55.482321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8303ee-eb34-486b-accb-24cd439cd190 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:55.577569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b4a29-90cf-4f45-b881-c2320c61d3ea admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:55.590261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15787249-9fa4-48d2-aacf-49a6a85aeac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:55.591098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787249-9fa4-48d2-aacf-49a6a85aeac3 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.591822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787249-9fa4-48d2-aacf-49a6a85aeac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.592137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787249-9fa4-48d2-aacf-49a6a85aeac3 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.593084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15787249-9fa4-48d2-aacf-49a6a85aeac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:55.596052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15787249-9fa4-48d2-aacf-49a6a85aeac3 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:55.634922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 749/1492] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:55 2022] POST /identity/v3/users => generated 285 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:55.636963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589799f-aeff-4592-98a8-0c8e7c848d80 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.637252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589799f-aeff-4592-98a8-0c8e7c848d80 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.637627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589799f-aeff-4592-98a8-0c8e7c848d80 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.639716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 750/1493] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:40:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:40:55.641501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe3340-822b-4e16-9144-b7d3b4fe2cc7 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.641785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe3340-822b-4e16-9144-b7d3b4fe2cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.642134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe3340-822b-4e16-9144-b7d3b4fe2cc7 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.644146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 751/1494] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:40:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:40:55.645326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe3340-822b-4e16-9144-b7d3b4fe2cc7 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:55.665466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe3340-822b-4e16-9144-b7d3b4fe2cc7 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:55.673165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5d655-b500-40df-850a-cabb2e3bf078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:55.674425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5d655-b500-40df-850a-cabb2e3bf078 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.674664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5d655-b500-40df-850a-cabb2e3bf078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.675025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5d655-b500-40df-850a-cabb2e3bf078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.676165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5d655-b500-40df-850a-cabb2e3bf078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:55.681631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5d655-b500-40df-850a-cabb2e3bf078 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:55.688575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117b1f4-b8cd-48cf-8c2a-99286e274612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:55.690216 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117b1f4-b8cd-48cf-8c2a-99286e274612 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.690510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117b1f4-b8cd-48cf-8c2a-99286e274612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.690776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117b1f4-b8cd-48cf-8c2a-99286e274612 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.691442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117b1f4-b8cd-48cf-8c2a-99286e274612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:55.693571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117b1f4-b8cd-48cf-8c2a-99286e274612 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:55.701773 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 752/1495] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:55.703078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5d655-b500-40df-850a-cabb2e3bf078 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:55.718904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5d655-b500-40df-850a-cabb2e3bf078 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:55.725226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf71578-cf1f-4674-808f-6ef09e98e8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:55.726643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf71578-cf1f-4674-808f-6ef09e98e8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.727163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf71578-cf1f-4674-808f-6ef09e98e8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.727740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf71578-cf1f-4674-808f-6ef09e98e8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.728951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf71578-cf1f-4674-808f-6ef09e98e8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:55.732430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf71578-cf1f-4674-808f-6ef09e98e8b4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:55.756013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 753/1496] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:55.758370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf71578-cf1f-4674-808f-6ef09e98e8b4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:55.772274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf71578-cf1f-4674-808f-6ef09e98e8b4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:55.780341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0099ae-52e4-44b2-8fb8-2cfb47b7d722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:55.781551 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0099ae-52e4-44b2-8fb8-2cfb47b7d722 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.782281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0099ae-52e4-44b2-8fb8-2cfb47b7d722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.783040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0099ae-52e4-44b2-8fb8-2cfb47b7d722 admin admin] PATH_INFO: `/v3/projects/d2ae03e2782a40e8b70d0b6396479b34/users/4265cd5cfeb941b0a9abed092fe37e6c/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.799602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 744/1497] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:55 2022] POST /identity/v3/users => generated 299 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:55.807953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0099ae-52e4-44b2-8fb8-2cfb47b7d722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ae03e2782a40e8b70d0b6396479b34, user_id=4265cd5cfeb941b0a9abed092fe37e6c, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:55.810117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117b1f4-b8cd-48cf-8c2a-99286e274612 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:55.825653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0099ae-52e4-44b2-8fb8-2cfb47b7d722 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:55.845148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117b1f4-b8cd-48cf-8c2a-99286e274612 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:55.856072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebb0a5-f3cb-4929-b315-3cfe2547159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:55.856707 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb0a5-f3cb-4929-b315-3cfe2547159e admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.857014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb0a5-f3cb-4929-b315-3cfe2547159e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.857331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb0a5-f3cb-4929-b315-3cfe2547159e admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.858010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ebb0a5-f3cb-4929-b315-3cfe2547159e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:55.863436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ebb0a5-f3cb-4929-b315-3cfe2547159e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:55.877071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 754/1498] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:55 2022] PUT /identity/v3/projects/d2ae03e2782a40e8b70d0b6396479b34/users/4265cd5cfeb941b0a9abed092fe37e6c/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:55.880660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af36322-f3e6-4d9a-971d-a5f6b6bf1359 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.881533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af36322-f3e6-4d9a-971d-a5f6b6bf1359 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.882204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af36322-f3e6-4d9a-971d-a5f6b6bf1359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.893074 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 745/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:55.893600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa3f56-95b2-4ee3-b196-91c5654d2781 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:55.901319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa3f56-95b2-4ee3-b196-91c5654d2781 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:55.901683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa3f56-95b2-4ee3-b196-91c5654d2781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:55.925972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6af36322-f3e6-4d9a-971d-a5f6b6bf1359 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:55.927404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6af36322-f3e6-4d9a-971d-a5f6b6bf1359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:56.005893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-c0fa3f56-95b2-4ee3-b196-91c5654d2781 None None] MFA Rules not processed for user `4265cd5cfeb941b0a9abed092fe37e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:56.007224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c0fa3f56-95b2-4ee3-b196-91c5654d2781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:56.074729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:56.142257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 755/1500] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:40:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:56.143370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af36322-f3e6-4d9a-971d-a5f6b6bf1359 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:56.158315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:56.170401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 746/1501] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:56.174499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8a5fc-9e4b-4f81-9e24-5e40b2abce02 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:56.176929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8a5fc-9e4b-4f81-9e24-5e40b2abce02 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:56.177144 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8a5fc-9e4b-4f81-9e24-5e40b2abce02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:56.234648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-80c8a5fc-9e4b-4f81-9e24-5e40b2abce02 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:56.236061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-80c8a5fc-9e4b-4f81-9e24-5e40b2abce02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:56.255201 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 747/1502] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:40:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:56.257657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8a5fc-9e4b-4f81-9e24-5e40b2abce02 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:56.318456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af36322-f3e6-4d9a-971d-a5f6b6bf1359 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:56.342720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760cbe-54df-4dce-ab03-dc1674595640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:56.346607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760cbe-54df-4dce-ab03-dc1674595640 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:56.347292 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760cbe-54df-4dce-ab03-dc1674595640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:56.351056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760cbe-54df-4dce-ab03-dc1674595640 admin admin] PATH_INFO: `/v3/projects/99dd2a2d63104f6ab51c1eb20f2bf81a/users/1c49851b2d964a91b5641b28a0a760ac/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:56.426802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8760cbe-54df-4dce-ab03-dc1674595640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99dd2a2d63104f6ab51c1eb20f2bf81a, user_id=1c49851b2d964a91b5641b28a0a760ac, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:56.442528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8760cbe-54df-4dce-ab03-dc1674595640 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:56.482136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8a5fc-9e4b-4f81-9e24-5e40b2abce02 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:56.492792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 756/1503] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:56 2022] PUT /identity/v3/projects/99dd2a2d63104f6ab51c1eb20f2bf81a/users/1c49851b2d964a91b5641b28a0a760ac/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:56.496706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760cbe-54df-4dce-ab03-dc1674595640 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:56.506434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a8760cbe-54df-4dce-ab03-dc1674595640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:56.689035 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd889d5-c359-45b5-8bae-fa58853e4df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:56.689890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd889d5-c359-45b5-8bae-fa58853e4df3 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:56.690224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd889d5-c359-45b5-8bae-fa58853e4df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:56.690477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd889d5-c359-45b5-8bae-fa58853e4df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:56.691472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd889d5-c359-45b5-8bae-fa58853e4df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:56.695680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd889d5-c359-45b5-8bae-fa58853e4df3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:56.707038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760cbe-54df-4dce-ab03-dc1674595640 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:56.730880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:56.731556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:56.731841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:56.732118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] PATH_INFO: `/v3/projects/ba9c292aafb94661bd46780013913954` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:56.733346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9c292aafb94661bd46780013913954)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:56.735593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:56.797766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 748/1504] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:56 2022] POST /identity/v3/projects => generated 369 bytes in 542 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:56.799104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd889d5-c359-45b5-8bae-fa58853e4df3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:56.802221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7cd889d5-c359-45b5-8bae-fa58853e4df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:56.864148 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9c292aafb94661bd46780013913954', 'request_id': 'req-d3a9bd20-a3e6-4638-b557-d51580b17627'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:56.883761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9c292aafb94661bd46780013913954', 'request_id': 'req-d3a9bd20-a3e6-4638-b557-d51580b17627'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:56.885836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:56.902373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9c292aafb94661bd46780013913954', 'request_id': 'req-d3a9bd20-a3e6-4638-b557-d51580b17627'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:56.932386 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] The token cache is being invalidate because project ba9c292aafb94661bd46780013913954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:40:56.932679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:56.934078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:40:56.936086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 757/1505] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:40:56 2022] DELETE /identity/v3/projects/ba9c292aafb94661bd46780013913954 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:56.936733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:56.941143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:56.990868 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd889d5-c359-45b5-8bae-fa58853e4df3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:57.033872 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:57.092015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9bd20-a3e6-4638-b557-d51580b17627 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:57.107797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f23b1-81b8-4793-bf62-d8d014bdc4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:57.108662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f23b1-81b8-4793-bf62-d8d014bdc4fb admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.109113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f23b1-81b8-4793-bf62-d8d014bdc4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.109559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f23b1-81b8-4793-bf62-d8d014bdc4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.110365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f23b1-81b8-4793-bf62-d8d014bdc4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:57.112749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f23b1-81b8-4793-bf62-d8d014bdc4fb admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:57.130691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42a03b-41a0-4bd7-82ca-c01b15d33430 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:57.132017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42a03b-41a0-4bd7-82ca-c01b15d33430 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.132770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42a03b-41a0-4bd7-82ca-c01b15d33430 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.133307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42a03b-41a0-4bd7-82ca-c01b15d33430 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.138172 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3a42a03b-41a0-4bd7-82ca-c01b15d33430 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:57.155838 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 758/1506] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:57.160134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f23b1-81b8-4793-bf62-d8d014bdc4fb admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:57.162787 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42a03b-41a0-4bd7-82ca-c01b15d33430 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:57.164282 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5c8f23b1-81b8-4793-bf62-d8d014bdc4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:57.166242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42a03b-41a0-4bd7-82ca-c01b15d33430 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:57.190517 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 749/1507] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:57.194898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec99ab3-cca2-489e-974c-404495792bb3 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.195390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec99ab3-cca2-489e-974c-404495792bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.195806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec99ab3-cca2-489e-974c-404495792bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.197624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f23b1-81b8-4793-bf62-d8d014bdc4fb admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:57.203907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450139d-b476-49ce-85c6-35ee4e1b71e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:57.204827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450139d-b476-49ce-85c6-35ee4e1b71e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.206003 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450139d-b476-49ce-85c6-35ee4e1b71e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.206003 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450139d-b476-49ce-85c6-35ee4e1b71e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.206416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450139d-b476-49ce-85c6-35ee4e1b71e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:57.209168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450139d-b476-49ce-85c6-35ee4e1b71e9 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:57.227022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-8ec99ab3-cca2-489e-974c-404495792bb3 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:57.229640 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8ec99ab3-cca2-489e-974c-404495792bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:57.251239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 750/1508] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:57.251864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 759/1509] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:57 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:57.253095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec99ab3-cca2-489e-974c-404495792bb3 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:57.262066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450139d-b476-49ce-85c6-35ee4e1b71e9 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:57.277645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec99ab3-cca2-489e-974c-404495792bb3 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:57.287742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae657d-1a66-4e01-a371-9a24725bad6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:57.289910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae657d-1a66-4e01-a371-9a24725bad6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.291573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae657d-1a66-4e01-a371-9a24725bad6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.293412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae657d-1a66-4e01-a371-9a24725bad6d admin admin] PATH_INFO: `/v3/projects/99dd2a2d63104f6ab51c1eb20f2bf81a/users/1c49851b2d964a91b5641b28a0a760ac/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.296819 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae657d-1a66-4e01-a371-9a24725bad6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=99dd2a2d63104f6ab51c1eb20f2bf81a, user_id=1c49851b2d964a91b5641b28a0a760ac, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:57.307765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae657d-1a66-4e01-a371-9a24725bad6d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:57.329430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450139d-b476-49ce-85c6-35ee4e1b71e9 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:57.354665 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 751/1510] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:57 2022] PUT /identity/v3/projects/99dd2a2d63104f6ab51c1eb20f2bf81a/users/1c49851b2d964a91b5641b28a0a760ac/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:57.358823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f360-69f9-4352-a8a8-4eedf30310bd None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.360433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f360-69f9-4352-a8a8-4eedf30310bd None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.361330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f360-69f9-4352-a8a8-4eedf30310bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.446036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-eb63f360-69f9-4352-a8a8-4eedf30310bd None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:57.449492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-eb63f360-69f9-4352-a8a8-4eedf30310bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:57.537790 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f0f75-efd4-4d76-8a3a-03911785cd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:57.538763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f0f75-efd4-4d76-8a3a-03911785cd47 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.539234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f0f75-efd4-4d76-8a3a-03911785cd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.539689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f0f75-efd4-4d76-8a3a-03911785cd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.543937 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f0f75-efd4-4d76-8a3a-03911785cd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:57.547510 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f0f75-efd4-4d76-8a3a-03911785cd47 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:57.579685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 760/1511] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:57.581528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c00f8-f767-40bf-adf0-2febb7639863 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.581937 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c00f8-f767-40bf-adf0-2febb7639863 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.582208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c00f8-f767-40bf-adf0-2febb7639863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.636172 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 752/1512] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:57.637665 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63f360-69f9-4352-a8a8-4eedf30310bd None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:57.638986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-eb63f360-69f9-4352-a8a8-4eedf30310bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:57.664629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-be6c00f8-f767-40bf-adf0-2febb7639863 None None] MFA Rules not processed for user `1c49851b2d964a91b5641b28a0a760ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:57.667285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-be6c00f8-f767-40bf-adf0-2febb7639863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:57.799518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63f360-69f9-4352-a8a8-4eedf30310bd None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:57.811855 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff741dd4-2361-47e9-a1b6-a002bcbb7b5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:57.812498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff741dd4-2361-47e9-a1b6-a002bcbb7b5d service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.812786 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff741dd4-2361-47e9-a1b6-a002bcbb7b5d service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.813076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff741dd4-2361-47e9-a1b6-a002bcbb7b5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.822450 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ff741dd4-2361-47e9-a1b6-a002bcbb7b5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:57.844802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:57.847320 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff741dd4-2361-47e9-a1b6-a002bcbb7b5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:57.849308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff741dd4-2361-47e9-a1b6-a002bcbb7b5d service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:57.885535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 753/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:57.887403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff741dd4-2361-47e9-a1b6-a002bcbb7b5d service nova] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:57.926537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff741dd4-2361-47e9-a1b6-a002bcbb7b5d service nova] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:57.928182 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 761/1514] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:57.932695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c00f8-f767-40bf-adf0-2febb7639863 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:57.945795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7e24b-652f-4d8e-bf9e-bf8fb3ae9970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:57.946496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e24b-652f-4d8e-bf9e-bf8fb3ae9970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.946779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e24b-652f-4d8e-bf9e-bf8fb3ae9970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.947166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e24b-652f-4d8e-bf9e-bf8fb3ae9970 admin admin] PATH_INFO: `/v3/projects/35e0ea06d83a4661970955c3dda01ef4/users/ddc5e8eaf9014ccbae0ca868bbea9e65/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.967912 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c00f8-f767-40bf-adf0-2febb7639863 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:57.989317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bf69a-fb9a-488d-93b0-81e14df80e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:57.989317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf69a-fb9a-488d-93b0-81e14df80e0c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:57.989317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf69a-fb9a-488d-93b0-81e14df80e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:57.989317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bf69a-fb9a-488d-93b0-81e14df80e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:57.989726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bf69a-fb9a-488d-93b0-81e14df80e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:40:57.989726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bf69a-fb9a-488d-93b0-81e14df80e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:57.991019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:57.995190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bf69a-fb9a-488d-93b0-81e14df80e0c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:58.001868 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7e24b-652f-4d8e-bf9e-bf8fb3ae9970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e0ea06d83a4661970955c3dda01ef4, user_id=ddc5e8eaf9014ccbae0ca868bbea9e65, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:58.023465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7e24b-652f-4d8e-bf9e-bf8fb3ae9970 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:58.055631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 762/1515] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:40:58.060383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bf69a-fb9a-488d-93b0-81e14df80e0c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:58.070713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d56bf69a-fb9a-488d-93b0-81e14df80e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:58.072886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 754/1516] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:57 2022] PUT /identity/v3/projects/35e0ea06d83a4661970955c3dda01ef4/users/ddc5e8eaf9014ccbae0ca868bbea9e65/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:58.073771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7e24b-652f-4d8e-bf9e-bf8fb3ae9970 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:58.275277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7e24b-652f-4d8e-bf9e-bf8fb3ae9970 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:58.283569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5ddb5-9b81-4bfd-95d7-bc3a89f0b368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:58.284449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5ddb5-9b81-4bfd-95d7-bc3a89f0b368 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:58.284778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5ddb5-9b81-4bfd-95d7-bc3a89f0b368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:58.285022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5ddb5-9b81-4bfd-95d7-bc3a89f0b368 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:58.285718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5ddb5-9b81-4bfd-95d7-bc3a89f0b368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:58.287829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5ddb5-9b81-4bfd-95d7-bc3a89f0b368 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:58.302064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bf69a-fb9a-488d-93b0-81e14df80e0c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:58.346140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 755/1517] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:58 2022] POST /identity/v3/projects => generated 369 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:58.347032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5ddb5-9b81-4bfd-95d7-bc3a89f0b368 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:58.377905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3864b3-2afb-4dca-8b70-ca59e4fa90ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:58.378411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3864b3-2afb-4dca-8b70-ca59e4fa90ae service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:58.383180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3864b3-2afb-4dca-8b70-ca59e4fa90ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:58.383546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3864b3-2afb-4dca-8b70-ca59e4fa90ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:58.444176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5ddb5-9b81-4bfd-95d7-bc3a89f0b368 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:58.467168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd3951-bf93-43a1-be09-a73071d1f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:58.467925 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd3951-bf93-43a1-be09-a73071d1f734 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:58.469302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd3951-bf93-43a1-be09-a73071d1f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:58.470600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd3951-bf93-43a1-be09-a73071d1f734 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:58.471698 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd3951-bf93-43a1-be09-a73071d1f734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:58.478178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd3951-bf93-43a1-be09-a73071d1f734 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:58.522464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 756/1518] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:58.524430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd3951-bf93-43a1-be09-a73071d1f734 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:58.532515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3864b3-2afb-4dca-8b70-ca59e4fa90ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:58.538556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3864b3-2afb-4dca-8b70-ca59e4fa90ae service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:58.560346 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd3951-bf93-43a1-be09-a73071d1f734 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:58.572778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174eaa4-3a9a-43ca-8c4c-096025679d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:58.573385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 763/1519] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:58.574701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3864b3-2afb-4dca-8b70-ca59e4fa90ae service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:58.576732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174eaa4-3a9a-43ca-8c4c-096025679d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:58.577965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174eaa4-3a9a-43ca-8c4c-096025679d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:58.579583 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174eaa4-3a9a-43ca-8c4c-096025679d72 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:58.581128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174eaa4-3a9a-43ca-8c4c-096025679d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:58.596059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174eaa4-3a9a-43ca-8c4c-096025679d72 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:58.612238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3864b3-2afb-4dca-8b70-ca59e4fa90ae service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:58.623676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3886f-ab03-4d41-88e1-8f81516d5711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:58.625753 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3886f-ab03-4d41-88e1-8f81516d5711 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:58.626264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3886f-ab03-4d41-88e1-8f81516d5711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:58.626622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3886f-ab03-4d41-88e1-8f81516d5711 admin admin] PATH_INFO: `/v3/projects/35e0ea06d83a4661970955c3dda01ef4/users/ddc5e8eaf9014ccbae0ca868bbea9e65/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:58.632491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3886f-ab03-4d41-88e1-8f81516d5711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e0ea06d83a4661970955c3dda01ef4, user_id=ddc5e8eaf9014ccbae0ca868bbea9e65, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:58.655347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 757/1520] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:58 2022] POST /identity/v3/users => generated 285 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:58.656412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3886f-ab03-4d41-88e1-8f81516d5711 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:58.658264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174eaa4-3a9a-43ca-8c4c-096025679d72 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:58.671618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174eaa4-3a9a-43ca-8c4c-096025679d72 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:58.693130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 764/1521] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:58 2022] PUT /identity/v3/projects/35e0ea06d83a4661970955c3dda01ef4/users/ddc5e8eaf9014ccbae0ca868bbea9e65/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:58.697915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa42420-d620-4f32-89d3-a9db84019736 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:58.699381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa42420-d620-4f32-89d3-a9db84019736 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:58.699914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa42420-d620-4f32-89d3-a9db84019736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:58.739508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-5aa42420-d620-4f32-89d3-a9db84019736 None None] MFA Rules not processed for user `ddc5e8eaf9014ccbae0ca868bbea9e65`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:58.742250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5aa42420-d620-4f32-89d3-a9db84019736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:58.770703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:58.772993 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:58.790785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f6d04-7262-4bac-b584-9b030a58f412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:58.791747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6d04-7262-4bac-b584-9b030a58f412 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:58.792613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6d04-7262-4bac-b584-9b030a58f412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:58.793110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6d04-7262-4bac-b584-9b030a58f412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:58.794189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f6d04-7262-4bac-b584-9b030a58f412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:58.796473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f6d04-7262-4bac-b584-9b030a58f412 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:58.811649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 758/1522] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:40:58.816283 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f6d04-7262-4bac-b584-9b030a58f412 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:58.857618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f6d04-7262-4bac-b584-9b030a58f412 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:58.865838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47317a37-8fc5-4303-b294-d036fff8cc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:58.867465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47317a37-8fc5-4303-b294-d036fff8cc9a admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:58.867920 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47317a37-8fc5-4303-b294-d036fff8cc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:58.868462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47317a37-8fc5-4303-b294-d036fff8cc9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:58.869487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47317a37-8fc5-4303-b294-d036fff8cc9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:58.871619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47317a37-8fc5-4303-b294-d036fff8cc9a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:58.894285 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 759/1523] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:58.896503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47317a37-8fc5-4303-b294-d036fff8cc9a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:58.914034 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47317a37-8fc5-4303-b294-d036fff8cc9a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:58.921775 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714977b-1841-42a0-9027-1c9c75768ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:58.923173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714977b-1841-42a0-9027-1c9c75768ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:58.923609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714977b-1841-42a0-9027-1c9c75768ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:58.924037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714977b-1841-42a0-9027-1c9c75768ab6 admin admin] PATH_INFO: `/v3/projects/eb97c7d7566d4a9ab33d0d204dba62fd/users/9b69338373ef4b51812a4bb56acd53a6/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:58.962853 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714977b-1841-42a0-9027-1c9c75768ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb97c7d7566d4a9ab33d0d204dba62fd, user_id=9b69338373ef4b51812a4bb56acd53a6, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:58.968942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714977b-1841-42a0-9027-1c9c75768ab6 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:58.988670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 765/1524] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:40:59.006297 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa42420-d620-4f32-89d3-a9db84019736 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:59.017563 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5aa42420-d620-4f32-89d3-a9db84019736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:59.024951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 760/1525] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:58 2022] PUT /identity/v3/projects/eb97c7d7566d4a9ab33d0d204dba62fd/users/9b69338373ef4b51812a4bb56acd53a6/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:40:59.031379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abfc9c-e496-4f49-94ef-cd23407397e4 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:59.031890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abfc9c-e496-4f49-94ef-cd23407397e4 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:59.032723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abfc9c-e496-4f49-94ef-cd23407397e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:59.117827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-26abfc9c-e496-4f49-94ef-cd23407397e4 None None] MFA Rules not processed for user `9b69338373ef4b51812a4bb56acd53a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:59.119058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-26abfc9c-e496-4f49-94ef-cd23407397e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:59.154607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa42420-d620-4f32-89d3-a9db84019736 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:59.233658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 761/1526] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:59.238907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abfc9c-e496-4f49-94ef-cd23407397e4 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:59.301518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c427a9e-4a7d-4b2a-80f0-90e0f7ba9c56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:59.302161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c427a9e-4a7d-4b2a-80f0-90e0f7ba9c56 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:59.302684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c427a9e-4a7d-4b2a-80f0-90e0f7ba9c56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:59.302684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c427a9e-4a7d-4b2a-80f0-90e0f7ba9c56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:59.382105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abfc9c-e496-4f49-94ef-cd23407397e4 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:59.389902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee8341-b122-49cd-94d9-b6ee7878f616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:59.391002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee8341-b122-49cd-94d9-b6ee7878f616 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:59.391408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee8341-b122-49cd-94d9-b6ee7878f616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:59.391776 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee8341-b122-49cd-94d9-b6ee7878f616 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:59.392533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee8341-b122-49cd-94d9-b6ee7878f616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:59.394647 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee8341-b122-49cd-94d9-b6ee7878f616 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:59.441752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 762/1527] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:59 2022] POST /identity/v3/projects => generated 367 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:59.442820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee8341-b122-49cd-94d9-b6ee7878f616 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:59.472938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c427a9e-4a7d-4b2a-80f0-90e0f7ba9c56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:59.474515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c427a9e-4a7d-4b2a-80f0-90e0f7ba9c56 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:59.490668 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 766/1528] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:40:59.492198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de5fed-1df2-4c3f-b9e9-f0360886b579 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:59.492684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de5fed-1df2-4c3f-b9e9-f0360886b579 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:59.493029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de5fed-1df2-4c3f-b9e9-f0360886b579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:59.503386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:59.595950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee8341-b122-49cd-94d9-b6ee7878f616 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:59.611123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6ccba-8818-45d9-a71b-71ba8a06182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:59.611890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6ccba-8818-45d9-a71b-71ba8a06182b admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:59.612272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6ccba-8818-45d9-a71b-71ba8a06182b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:59.612809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6ccba-8818-45d9-a71b-71ba8a06182b admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:59.613769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6ccba-8818-45d9-a71b-71ba8a06182b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:59.616116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6ccba-8818-45d9-a71b-71ba8a06182b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:59.666131 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 763/1529] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:59 2022] POST /identity/v3/users => generated 290 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:59.668345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6ccba-8818-45d9-a71b-71ba8a06182b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:59.685604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-b0de5fed-1df2-4c3f-b9e9-f0360886b579 None None] MFA Rules not processed for user `eb41f09835414ed1a585f2485bb442cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:40:59.688237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b0de5fed-1df2-4c3f-b9e9-f0360886b579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:59.694857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:40:59.770049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6ccba-8818-45d9-a71b-71ba8a06182b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:59.774033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1bd2e-68d8-40bf-8708-e33791ad837d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:59.775178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1bd2e-68d8-40bf-8708-e33791ad837d service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:59.775178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1bd2e-68d8-40bf-8708-e33791ad837d service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:59.775509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1bd2e-68d8-40bf-8708-e33791ad837d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:59.778743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d1c1bd2e-68d8-40bf-8708-e33791ad837d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:59.811591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 767/1530] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:40:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:40:59.812708 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de5fed-1df2-4c3f-b9e9-f0360886b579 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:59.825233 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de5fed-1df2-4c3f-b9e9-f0360886b579 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:59.830926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6ffed-b5bf-426d-962b-3822fce7fa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:59.831542 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6ffed-b5bf-426d-962b-3822fce7fa92 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:59.832260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6ffed-b5bf-426d-962b-3822fce7fa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:59.832559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6ffed-b5bf-426d-962b-3822fce7fa92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:59.833865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6ffed-b5bf-426d-962b-3822fce7fa92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:59.835472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6ffed-b5bf-426d-962b-3822fce7fa92 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:59.851017 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 768/1531] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:40:59.851200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1bd2e-68d8-40bf-8708-e33791ad837d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:59.851314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6ffed-b5bf-426d-962b-3822fce7fa92 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:59.851401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1bd2e-68d8-40bf-8708-e33791ad837d service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:59.861904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6ffed-b5bf-426d-962b-3822fce7fa92 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:59.865441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 764/1532] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:59.866222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10afb60-7f07-4426-9c1f-6e8614e585cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:59.866385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1bd2e-68d8-40bf-8708-e33791ad837d service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:59.867135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10afb60-7f07-4426-9c1f-6e8614e585cd service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:59.867721 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10afb60-7f07-4426-9c1f-6e8614e585cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:59.867830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d1c1bd2e-68d8-40bf-8708-e33791ad837d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:59.868219 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10afb60-7f07-4426-9c1f-6e8614e585cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:59.870461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e10afb60-7f07-4426-9c1f-6e8614e585cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:40:59.896532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1bd2e-68d8-40bf-8708-e33791ad837d service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:40:59.902882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca46f7-1db0-4430-8586-0e2116ca3923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:40:59.903497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca46f7-1db0-4430-8586-0e2116ca3923 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:40:59.903774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca46f7-1db0-4430-8586-0e2116ca3923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:40:59.904184 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca46f7-1db0-4430-8586-0e2116ca3923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:40:59.905095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca46f7-1db0-4430-8586-0e2116ca3923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:59.907017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca46f7-1db0-4430-8586-0e2116ca3923 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:59.944532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 765/1533] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:40:59 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:40:59.949638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca46f7-1db0-4430-8586-0e2116ca3923 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:40:59.978703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10afb60-7f07-4426-9c1f-6e8614e585cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:40:59.981733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10afb60-7f07-4426-9c1f-6e8614e585cd service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:40:59.994880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 769/1534] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:40:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:40:59.996077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10afb60-7f07-4426-9c1f-6e8614e585cd service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:00.009556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10afb60-7f07-4426-9c1f-6e8614e585cd service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:00.031018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:00.062205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca46f7-1db0-4430-8586-0e2116ca3923 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:00.073129 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798ece6-070a-40a5-87d8-a08b3ec26869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:00.074902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798ece6-070a-40a5-87d8-a08b3ec26869 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:00.075302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798ece6-070a-40a5-87d8-a08b3ec26869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:00.075668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798ece6-070a-40a5-87d8-a08b3ec26869 admin admin] PATH_INFO: `/v3/projects/68ad9eefe2a747cba281ec178da52e24/users/ab51bb80af91444b96a9c9e898b5c2e9/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:00.099749 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798ece6-070a-40a5-87d8-a08b3ec26869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ad9eefe2a747cba281ec178da52e24, user_id=ab51bb80af91444b96a9c9e898b5c2e9, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:00.103343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798ece6-070a-40a5-87d8-a08b3ec26869 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:00.110403 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d3f42-18db-458f-8a15-421ee5a982a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:00.111294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d3f42-18db-458f-8a15-421ee5a982a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:00.111578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d3f42-18db-458f-8a15-421ee5a982a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:00.111880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d3f42-18db-458f-8a15-421ee5a982a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:00.112520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d3f42-18db-458f-8a15-421ee5a982a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:00.118809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d3f42-18db-458f-8a15-421ee5a982a4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:00.143590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 766/1535] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:40:59 2022] PUT /identity/v3/projects/68ad9eefe2a747cba281ec178da52e24/users/ab51bb80af91444b96a9c9e898b5c2e9/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:00.145356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798ece6-070a-40a5-87d8-a08b3ec26869 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:00.170481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 770/1536] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:40:59 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:00.170481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d3f42-18db-458f-8a15-421ee5a982a4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:00.191189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d3f42-18db-458f-8a15-421ee5a982a4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:00.260928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:00.265003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798ece6-070a-40a5-87d8-a08b3ec26869 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:00.277365 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d315933-fe95-4a3a-98f7-ba890494b0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:00.278290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d315933-fe95-4a3a-98f7-ba890494b0aa admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:00.278580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d315933-fe95-4a3a-98f7-ba890494b0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:00.278863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d315933-fe95-4a3a-98f7-ba890494b0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:00.281449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d315933-fe95-4a3a-98f7-ba890494b0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:00.283200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d315933-fe95-4a3a-98f7-ba890494b0aa admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:00.302759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 767/1537] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:00.320116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d315933-fe95-4a3a-98f7-ba890494b0aa admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:00.356087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d315933-fe95-4a3a-98f7-ba890494b0aa admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:00.375136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e591f-e36b-45c4-8994-78c3e29f3109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:00.376376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e591f-e36b-45c4-8994-78c3e29f3109 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:00.376376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e591f-e36b-45c4-8994-78c3e29f3109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:00.376551 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e591f-e36b-45c4-8994-78c3e29f3109 admin admin] PATH_INFO: `/v3/projects/68ad9eefe2a747cba281ec178da52e24/users/ab51bb80af91444b96a9c9e898b5c2e9/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:00.380400 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e591f-e36b-45c4-8994-78c3e29f3109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ad9eefe2a747cba281ec178da52e24, user_id=ab51bb80af91444b96a9c9e898b5c2e9, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:00.389532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cbefd-30cd-443e-9f4c-5659290d240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:00.395689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cbefd-30cd-443e-9f4c-5659290d240c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:00.395689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cbefd-30cd-443e-9f4c-5659290d240c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:00.395689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cbefd-30cd-443e-9f4c-5659290d240c admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:00.395689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cbefd-30cd-443e-9f4c-5659290d240c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:00.395689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cbefd-30cd-443e-9f4c-5659290d240c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:00.398716 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e591f-e36b-45c4-8994-78c3e29f3109 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:00.427696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 771/1538] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:00.434589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cbefd-30cd-443e-9f4c-5659290d240c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:00.438561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 768/1539] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:00 2022] PUT /identity/v3/projects/68ad9eefe2a747cba281ec178da52e24/users/ab51bb80af91444b96a9c9e898b5c2e9/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:00.451314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7eab1-cbca-4158-b7da-a0ab6c980998 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:00.452905 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7eab1-cbca-4158-b7da-a0ab6c980998 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:00.453308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7eab1-cbca-4158-b7da-a0ab6c980998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:00.467086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cbefd-30cd-443e-9f4c-5659290d240c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:00.542177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7cf7eab1-cbca-4158-b7da-a0ab6c980998 None None] MFA Rules not processed for user `ab51bb80af91444b96a9c9e898b5c2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:00.544345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7cf7eab1-cbca-4158-b7da-a0ab6c980998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:00.617449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00634f-767d-4ec6-9062-87a6f62de75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:00.618486 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00634f-767d-4ec6-9062-87a6f62de75f admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:00.619697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00634f-767d-4ec6-9062-87a6f62de75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:00.620040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00634f-767d-4ec6-9062-87a6f62de75f admin admin] PATH_INFO: `/v3/projects/3e3b62e3524a4b9d809ff96769d9edf8/users/77e7142139174f29a2d4ee837c3195a0/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:00.643171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00634f-767d-4ec6-9062-87a6f62de75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3b62e3524a4b9d809ff96769d9edf8, user_id=77e7142139174f29a2d4ee837c3195a0, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:00.646156 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00634f-767d-4ec6-9062-87a6f62de75f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:00.660091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 769/1540] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:00.674009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 772/1541] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:00 2022] PUT /identity/v3/projects/3e3b62e3524a4b9d809ff96769d9edf8/users/77e7142139174f29a2d4ee837c3195a0/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:00.677983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7eab1-cbca-4158-b7da-a0ab6c980998 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:00.678455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00634f-767d-4ec6-9062-87a6f62de75f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:00.728288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:00.740280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7eab1-cbca-4158-b7da-a0ab6c980998 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:00.744511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00634f-767d-4ec6-9062-87a6f62de75f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:00.749735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159ea33-4490-41c9-896b-324a92083207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:00.750341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159ea33-4490-41c9-896b-324a92083207 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:00.750616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159ea33-4490-41c9-896b-324a92083207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:00.750894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159ea33-4490-41c9-896b-324a92083207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:00.751526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159ea33-4490-41c9-896b-324a92083207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:00.753166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159ea33-4490-41c9-896b-324a92083207 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:00.767078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 773/1542] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:00.768438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159ea33-4490-41c9-896b-324a92083207 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:00.778517 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159ea33-4490-41c9-896b-324a92083207 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:00.783260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48582f26-e13b-4bae-b27a-e076cfbe9f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:00.784135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48582f26-e13b-4bae-b27a-e076cfbe9f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:00.784415 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48582f26-e13b-4bae-b27a-e076cfbe9f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:00.784684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48582f26-e13b-4bae-b27a-e076cfbe9f54 admin admin] PATH_INFO: `/v3/projects/3e3b62e3524a4b9d809ff96769d9edf8/users/77e7142139174f29a2d4ee837c3195a0/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:00.786738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48582f26-e13b-4bae-b27a-e076cfbe9f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3b62e3524a4b9d809ff96769d9edf8, user_id=77e7142139174f29a2d4ee837c3195a0, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:00.789410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48582f26-e13b-4bae-b27a-e076cfbe9f54 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:00.791401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8d72a-336c-4d59-aa76-174ba90a2856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:00.792020 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d72a-336c-4d59-aa76-174ba90a2856 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:00.792358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d72a-336c-4d59-aa76-174ba90a2856 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:00.793179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d72a-336c-4d59-aa76-174ba90a2856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:00.801522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 774/1543] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:00 2022] PUT /identity/v3/projects/3e3b62e3524a4b9d809ff96769d9edf8/users/77e7142139174f29a2d4ee837c3195a0/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:00.860678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8d72a-336c-4d59-aa76-174ba90a2856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:00.862170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8d72a-336c-4d59-aa76-174ba90a2856 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:00.872604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 770/1544] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:02.975284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cea5f5-df74-45fd-afd7-187902b273c7 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:02.975886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cea5f5-df74-45fd-afd7-187902b273c7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:02.976000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cea5f5-df74-45fd-afd7-187902b273c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:03.039596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-68cea5f5-df74-45fd-afd7-187902b273c7 None None] MFA Rules not processed for user `77e7142139174f29a2d4ee837c3195a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:03.041135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-68cea5f5-df74-45fd-afd7-187902b273c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:03.059589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:03.097313 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8d72a-336c-4d59-aa76-174ba90a2856 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:03.132483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8d72a-336c-4d59-aa76-174ba90a2856 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:03.210730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 775/1545] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:03.223432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cea5f5-df74-45fd-afd7-187902b273c7 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:03.275943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:03.277598 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:03.278311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:03.278905 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] PATH_INFO: `/v3/users/9b69338373ef4b51812a4bb56acd53a6` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:03.289264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b69338373ef4b51812a4bb56acd53a6)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:03.293234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:03.309562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cea5f5-df74-45fd-afd7-187902b273c7 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:03.314438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5943376-061c-4a72-b37a-326cbf8060b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:03.315497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5943376-061c-4a72-b37a-326cbf8060b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:03.315783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5943376-061c-4a72-b37a-326cbf8060b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:03.316543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5943376-061c-4a72-b37a-326cbf8060b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:03.377702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b69338373ef4b51812a4bb56acd53a6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:03.385510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:03.390338 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b69338373ef4b51812a4bb56acd53a6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:03.391444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b69338373ef4b51812a4bb56acd53a6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:03.398304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5943376-061c-4a72-b37a-326cbf8060b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:03.400010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5943376-061c-4a72-b37a-326cbf8060b1 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:03.411088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b69338373ef4b51812a4bb56acd53a6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:03.422676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b69338373ef4b51812a4bb56acd53a6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:03.425380 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 776/1546] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:41:03.435236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 771/1547] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:03 2022] DELETE /identity/v3/users/9b69338373ef4b51812a4bb56acd53a6 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:03.440283 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e47f2c-eeb8-4258-b319-b8985efc829f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:03.440576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e47f2c-eeb8-4258-b319-b8985efc829f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:03.440872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e47f2c-eeb8-4258-b319-b8985efc829f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:03.464832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-98e47f2c-eeb8-4258-b319-b8985efc829f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:03.466547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-98e47f2c-eeb8-4258-b319-b8985efc829f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:03.571653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 777/1548] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:03.584028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:03.587936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:03.739257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37406c-5eab-4c43-b547-6681150eba26 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:03.744063 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f263d-587a-4f41-8aa3-3b12c0139ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:03.744883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f263d-587a-4f41-8aa3-3b12c0139ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:03.746559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f263d-587a-4f41-8aa3-3b12c0139ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:03.746559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f263d-587a-4f41-8aa3-3b12c0139ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:03.759499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f263d-587a-4f41-8aa3-3b12c0139ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:03.762873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f263d-587a-4f41-8aa3-3b12c0139ab2 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:03.779100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 772/1549] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:04.324527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e47f2c-eeb8-4258-b319-b8985efc829f None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:04.326912 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-98e47f2c-eeb8-4258-b319-b8985efc829f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:04.342260 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f263d-587a-4f41-8aa3-3b12c0139ab2 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:04.344307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-632f263d-587a-4f41-8aa3-3b12c0139ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:04.346491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e47f2c-eeb8-4258-b319-b8985efc829f None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:04.351826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:04.352495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:04.352751 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:04.353063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] PATH_INFO: `/v3/projects/eb97c7d7566d4a9ab33d0d204dba62fd` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:04.354224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb97c7d7566d4a9ab33d0d204dba62fd)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:04.356716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:04.358600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:04.359077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f263d-587a-4f41-8aa3-3b12c0139ab2 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:04.364385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:04.365162 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:04.365628 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:04.365971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] PATH_INFO: `/v3/users/4265cd5cfeb941b0a9abed092fe37e6c` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:04.367165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4265cd5cfeb941b0a9abed092fe37e6c)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:04.369100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:04.409690 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb97c7d7566d4a9ab33d0d204dba62fd', 'request_id': 'req-241b20d0-5bdc-421e-b7f1-396e776f2677'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.415652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb97c7d7566d4a9ab33d0d204dba62fd', 'request_id': 'req-241b20d0-5bdc-421e-b7f1-396e776f2677'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.420244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4265cd5cfeb941b0a9abed092fe37e6c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.424052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb97c7d7566d4a9ab33d0d204dba62fd', 'request_id': 'req-241b20d0-5bdc-421e-b7f1-396e776f2677'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.428378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4265cd5cfeb941b0a9abed092fe37e6c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.429108 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4265cd5cfeb941b0a9abed092fe37e6c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.435886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] The token cache is being invalidate because project eb97c7d7566d4a9ab33d0d204dba62fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:04.436172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.436874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.438546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 778/1550] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:04 2022] DELETE /identity/v3/projects/eb97c7d7566d4a9ab33d0d204dba62fd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:04.442342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:04.443677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4265cd5cfeb941b0a9abed092fe37e6c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.443811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:04.454851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-65eaa188-46b0-4c42-b7e4-94bb4bc30f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4265cd5cfeb941b0a9abed092fe37e6c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.468945 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 773/1551] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:04 2022] DELETE /identity/v3/users/4265cd5cfeb941b0a9abed092fe37e6c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:04.470367 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940bd67-29af-4e34-a9f8-b8279f5b932d None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:04.470654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940bd67-29af-4e34-a9f8-b8279f5b932d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:04.470952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940bd67-29af-4e34-a9f8-b8279f5b932d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:04.494629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a940bd67-29af-4e34-a9f8-b8279f5b932d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:04.496316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a940bd67-29af-4e34-a9f8-b8279f5b932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:04.550453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b20d0-5bdc-421e-b7f1-396e776f2677 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:04.571122 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:04.612810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 774/1552] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:04.644604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940bd67-29af-4e34-a9f8-b8279f5b932d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:04.646416 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a940bd67-29af-4e34-a9f8-b8279f5b932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:04.702413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:04.703082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:04.703359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:04.703632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] PATH_INFO: `/v3/users/ab51bb80af91444b96a9c9e898b5c2e9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:04.708260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab51bb80af91444b96a9c9e898b5c2e9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:04.711927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:04.791486 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940bd67-29af-4e34-a9f8-b8279f5b932d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:04.797568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f4808-1cc9-4293-a62e-e81f9a1c8e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:04.798461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f4808-1cc9-4293-a62e-e81f9a1c8e03 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:04.798837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f4808-1cc9-4293-a62e-e81f9a1c8e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:04.799272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f4808-1cc9-4293-a62e-e81f9a1c8e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:04.812596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f4808-1cc9-4293-a62e-e81f9a1c8e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:04.814389 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f4808-1cc9-4293-a62e-e81f9a1c8e03 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:04.816034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:04.824811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab51bb80af91444b96a9c9e898b5c2e9'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.828748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 775/1553] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:04.865390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab51bb80af91444b96a9c9e898b5c2e9'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.865781 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab51bb80af91444b96a9c9e898b5c2e9'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.878231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab51bb80af91444b96a9c9e898b5c2e9'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.880040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab51bb80af91444b96a9c9e898b5c2e9'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:04.913613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 779/1554] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:04 2022] DELETE /identity/v3/users/ab51bb80af91444b96a9c9e898b5c2e9 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:05.027446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f4808-1cc9-4293-a62e-e81f9a1c8e03 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:05.029649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e06f4808-1cc9-4293-a62e-e81f9a1c8e03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:05.122123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:05.123556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:05.141393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f4808-1cc9-4293-a62e-e81f9a1c8e03 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:05.257374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cce276-6dd1-4f9e-8406-c0618b2dc6a2 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:05.263834 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e563c0-46fe-4042-82d5-d256bf425041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:05.265155 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e563c0-46fe-4042-82d5-d256bf425041 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:05.265671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e563c0-46fe-4042-82d5-d256bf425041 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:05.266702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e563c0-46fe-4042-82d5-d256bf425041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:05.269208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-60e563c0-46fe-4042-82d5-d256bf425041 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:05.288701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cce8e4-be89-4f67-8772-1680ae0613e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:05.289368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce8e4-be89-4f67-8772-1680ae0613e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:05.289727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce8e4-be89-4f67-8772-1680ae0613e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:05.289921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce8e4-be89-4f67-8772-1680ae0613e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:05.291720 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-92cce8e4-be89-4f67-8772-1680ae0613e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:05.369316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e563c0-46fe-4042-82d5-d256bf425041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:05.371589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e563c0-46fe-4042-82d5-d256bf425041 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:05.395138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 780/1555] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:41:05.399360 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445b74c-bd0a-403c-b416-fbf7345269c3 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:05.399919 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445b74c-bd0a-403c-b416-fbf7345269c3 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:05.401985 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445b74c-bd0a-403c-b416-fbf7345269c3 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:05.405684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 781/1556] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:41:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:41:05.419512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb56b03-6f45-4851-bf36-4a21aa1ac667 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:05.419801 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb56b03-6f45-4851-bf36-4a21aa1ac667 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:05.420122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb56b03-6f45-4851-bf36-4a21aa1ac667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:05.449067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cce8e4-be89-4f67-8772-1680ae0613e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:05.450552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cce8e4-be89-4f67-8772-1680ae0613e5 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:05.462929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:05.470616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-abb56b03-6f45-4851-bf36-4a21aa1ac667 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:05.472196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 776/1557] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:41:05.486382 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-abb56b03-6f45-4851-bf36-4a21aa1ac667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:05.655095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:05.691360 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 782/1558] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:41:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:05.846714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cce8e4-be89-4f67-8772-1680ae0613e5 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:05.848167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-92cce8e4-be89-4f67-8772-1680ae0613e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:05.870030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cce8e4-be89-4f67-8772-1680ae0613e5 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:05.885304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cae92b-20d4-44a9-a34d-d395d65b3dd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:05.888218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cae92b-20d4-44a9-a34d-d395d65b3dd4 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:05.888519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cae92b-20d4-44a9-a34d-d395d65b3dd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:05.888910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cae92b-20d4-44a9-a34d-d395d65b3dd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:05.905474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cae92b-20d4-44a9-a34d-d395d65b3dd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:05.907182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cae92b-20d4-44a9-a34d-d395d65b3dd4 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:05.921154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 777/1559] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:05.963943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb56b03-6f45-4851-bf36-4a21aa1ac667 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:05.966553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-abb56b03-6f45-4851-bf36-4a21aa1ac667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:06.065372 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb56b03-6f45-4851-bf36-4a21aa1ac667 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:06.070426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cae92b-20d4-44a9-a34d-d395d65b3dd4 service nova] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:06.070644 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:06.071267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:06.071541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:06.072125 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] PATH_INFO: `/v3/projects/d2ae03e2782a40e8b70d0b6396479b34` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:06.072571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a6cae92b-20d4-44a9-a34d-d395d65b3dd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:06.073898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ae03e2782a40e8b70d0b6396479b34)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:06.075834 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:06.110666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cae92b-20d4-44a9-a34d-d395d65b3dd4 service nova] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:06.128768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ae03e2782a40e8b70d0b6396479b34', 'request_id': 'req-7f5d96ec-49fa-46d3-b880-a765efde25ef'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.137521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ae03e2782a40e8b70d0b6396479b34', 'request_id': 'req-7f5d96ec-49fa-46d3-b880-a765efde25ef'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.146170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ae03e2782a40e8b70d0b6396479b34', 'request_id': 'req-7f5d96ec-49fa-46d3-b880-a765efde25ef'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.159149 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] The token cache is being invalidate because project d2ae03e2782a40e8b70d0b6396479b34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:06.159445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.160114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.162066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 783/1560] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:05 2022] DELETE /identity/v3/projects/d2ae03e2782a40e8b70d0b6396479b34 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:06.170562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:06.170562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:06.188268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:06.189086 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:06.189482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:06.189819 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] PATH_INFO: `/v3/projects/68ad9eefe2a747cba281ec178da52e24` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:06.191030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68ad9eefe2a747cba281ec178da52e24)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:06.193561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:06.256012 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ad9eefe2a747cba281ec178da52e24', 'request_id': 'req-ab8c547d-1503-4765-83da-a15add3f4ef2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.256379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d96ec-49fa-46d3-b880-a765efde25ef admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:06.263147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ad9eefe2a747cba281ec178da52e24', 'request_id': 'req-ab8c547d-1503-4765-83da-a15add3f4ef2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.272101 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ad9eefe2a747cba281ec178da52e24', 'request_id': 'req-ab8c547d-1503-4765-83da-a15add3f4ef2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.297566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] The token cache is being invalidate because project 68ad9eefe2a747cba281ec178da52e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:06.298119 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.298877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ab8c547d-1503-4765-83da-a15add3f4ef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.301461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 778/1561] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:06 2022] DELETE /identity/v3/projects/68ad9eefe2a747cba281ec178da52e24 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:06.306542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e25c7-716a-410c-8fdd-ec69d9699989 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:06.306842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e25c7-716a-410c-8fdd-ec69d9699989 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:06.307226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e25c7-716a-410c-8fdd-ec69d9699989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:06.307319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:06.330320 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7a5e25c7-716a-410c-8fdd-ec69d9699989 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:06.331715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7a5e25c7-716a-410c-8fdd-ec69d9699989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:06.339723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:06.341271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:06.341637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:06.341988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] PATH_INFO: `/v3/users/ddc5e8eaf9014ccbae0ca868bbea9e65` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:06.343298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddc5e8eaf9014ccbae0ca868bbea9e65)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:06.345460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:06.458770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 779/1562] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:06.462201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddc5e8eaf9014ccbae0ca868bbea9e65'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.474429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec44d4-babb-4db3-8980-38849b50e500 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:06.474708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec44d4-babb-4db3-8980-38849b50e500 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:06.477645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec44d4-babb-4db3-8980-38849b50e500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:06.506311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddc5e8eaf9014ccbae0ca868bbea9e65'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.508474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddc5e8eaf9014ccbae0ca868bbea9e65'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.532642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddc5e8eaf9014ccbae0ca868bbea9e65'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.533457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddc5e8eaf9014ccbae0ca868bbea9e65'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:06.537117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-2bec44d4-babb-4db3-8980-38849b50e500 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:06.539297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2bec44d4-babb-4db3-8980-38849b50e500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:06.552248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 780/1563] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:06.558484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec44d4-babb-4db3-8980-38849b50e500 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:06.560681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 784/1564] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:06 2022] DELETE /identity/v3/users/ddc5e8eaf9014ccbae0ca868bbea9e65 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:06.572877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec44d4-babb-4db3-8980-38849b50e500 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:06.579962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:06.751191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0966a64-6841-4df0-bbd5-73bd690cbef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:06.752270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0966a64-6841-4df0-bbd5-73bd690cbef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:06.753328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0966a64-6841-4df0-bbd5-73bd690cbef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:06.753882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0966a64-6841-4df0-bbd5-73bd690cbef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:06.754678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0966a64-6841-4df0-bbd5-73bd690cbef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:06.755093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0966a64-6841-4df0-bbd5-73bd690cbef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:06.757441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0966a64-6841-4df0-bbd5-73bd690cbef1 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:06.806066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:06.810589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 781/1565] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:06.825762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee654b-67f0-4694-ba75-f7beaadf65af admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:06.857796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d70f3f-0d84-47d8-b37e-7508681c8e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:06.860746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d70f3f-0d84-47d8-b37e-7508681c8e30 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:06.861500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d70f3f-0d84-47d8-b37e-7508681c8e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:06.862852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d70f3f-0d84-47d8-b37e-7508681c8e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:06.864733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d70f3f-0d84-47d8-b37e-7508681c8e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:06.869879 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d70f3f-0d84-47d8-b37e-7508681c8e30 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:07.001176 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 785/1566] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:06 2022] POST /identity/v3/projects => generated 409 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:07.004129 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0966a64-6841-4df0-bbd5-73bd690cbef1 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:07.026853 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0966a64-6841-4df0-bbd5-73bd690cbef1 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:07.106950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d70f3f-0d84-47d8-b37e-7508681c8e30 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:07.108835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-23d70f3f-0d84-47d8-b37e-7508681c8e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:07.221591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c4431-2c99-4081-b647-ae299ef61cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:07.224326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c4431-2c99-4081-b647-ae299ef61cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:07.224638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c4431-2c99-4081-b647-ae299ef61cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:07.225417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c4431-2c99-4081-b647-ae299ef61cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:07.227434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c4431-2c99-4081-b647-ae299ef61cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:07.231570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c4431-2c99-4081-b647-ae299ef61cc5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:07.296123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d70f3f-0d84-47d8-b37e-7508681c8e30 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:07.301769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 782/1567] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:07 2022] POST /identity/v3/users => generated 305 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:07.303479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c4431-2c99-4081-b647-ae299ef61cc5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:07.313333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:07.314040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:07.316240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:07.316541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] PATH_INFO: `/v3/projects/35e0ea06d83a4661970955c3dda01ef4` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:07.342072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e0ea06d83a4661970955c3dda01ef4)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:07.344299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:07.356105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c4431-2c99-4081-b647-ae299ef61cc5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:07.369610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e4781-3066-460a-ad1c-6878964d45dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:07.370284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e4781-3066-460a-ad1c-6878964d45dd admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:07.370565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e4781-3066-460a-ad1c-6878964d45dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:07.370832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e4781-3066-460a-ad1c-6878964d45dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:07.371656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e4781-3066-460a-ad1c-6878964d45dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:07.377904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e4781-3066-460a-ad1c-6878964d45dd admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:07.425768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 783/1568] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:07.432352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e4781-3066-460a-ad1c-6878964d45dd admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:07.434090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:07.457177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e4781-3066-460a-ad1c-6878964d45dd admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:07.458722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e0ea06d83a4661970955c3dda01ef4', 'request_id': 'req-4da099d9-2cc7-427f-9c22-56e938451f24'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:07.473113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e0ea06d83a4661970955c3dda01ef4', 'request_id': 'req-4da099d9-2cc7-427f-9c22-56e938451f24'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:07.487404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e0ea06d83a4661970955c3dda01ef4', 'request_id': 'req-4da099d9-2cc7-427f-9c22-56e938451f24'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:07.506142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] The token cache is being invalidate because project 35e0ea06d83a4661970955c3dda01ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:07.506453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:07.508046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4da099d9-2cc7-427f-9c22-56e938451f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:07.515629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 786/1569] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:07 2022] DELETE /identity/v3/projects/35e0ea06d83a4661970955c3dda01ef4 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:07.518017 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76ae16-1930-476c-b20b-9d4be80e9ff6 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:07.518522 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76ae16-1930-476c-b20b-9d4be80e9ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:07.519176 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76ae16-1930-476c-b20b-9d4be80e9ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:07.564785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-fb76ae16-1930-476c-b20b-9d4be80e9ff6 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:07.569966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-fb76ae16-1930-476c-b20b-9d4be80e9ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:07.654019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9ada1-3674-4869-adc7-ea090a4156b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:07.654670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9ada1-3674-4869-adc7-ea090a4156b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:07.654670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9ada1-3674-4869-adc7-ea090a4156b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:07.655029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9ada1-3674-4869-adc7-ea090a4156b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:07.655573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9ada1-3674-4869-adc7-ea090a4156b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:07.657244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9ada1-3674-4869-adc7-ea090a4156b2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:07.673866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 787/1570] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:07.679823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa95ca-49d1-40ea-8024-a0b216aa219e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:07.680197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa95ca-49d1-40ea-8024-a0b216aa219e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:07.680547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa95ca-49d1-40ea-8024-a0b216aa219e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:07.699792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 784/1571] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:07.701401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9ada1-3674-4869-adc7-ea090a4156b2 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:07.705197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2ce9ada1-3674-4869-adc7-ea090a4156b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:07.727633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9ada1-3674-4869-adc7-ea090a4156b2 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:07.734222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-f7fa95ca-49d1-40ea-8024-a0b216aa219e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:07.735189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f5c7b-6a9f-493d-8e8a-18bfa3ed9bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:07.735497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f7fa95ca-49d1-40ea-8024-a0b216aa219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:07.737120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f5c7b-6a9f-493d-8e8a-18bfa3ed9bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:07.737961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f5c7b-6a9f-493d-8e8a-18bfa3ed9bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:07.738673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f5c7b-6a9f-493d-8e8a-18bfa3ed9bd5 admin admin] PATH_INFO: `/v3/projects/48cfa127668f46f0a33edf6bb9266e1d/users/2ef4a1ca9ddc43f0b0a8271ed202178d/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:07.758901 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 788/1572] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:07.764005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa95ca-49d1-40ea-8024-a0b216aa219e None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:07.778757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f5c7b-6a9f-493d-8e8a-18bfa3ed9bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cfa127668f46f0a33edf6bb9266e1d, user_id=2ef4a1ca9ddc43f0b0a8271ed202178d, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:07.783271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa95ca-49d1-40ea-8024-a0b216aa219e None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:07.784276 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f5c7b-6a9f-493d-8e8a-18bfa3ed9bd5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:07.787229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:07.796809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396e6d7-f794-4d77-ac9d-aee3280a76c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:07.798020 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e6d7-f794-4d77-ac9d-aee3280a76c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:07.798614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e6d7-f794-4d77-ac9d-aee3280a76c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:07.801119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396e6d7-f794-4d77-ac9d-aee3280a76c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:07.802425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396e6d7-f794-4d77-ac9d-aee3280a76c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:07.805929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 785/1573] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:07 2022] PUT /identity/v3/projects/48cfa127668f46f0a33edf6bb9266e1d/users/2ef4a1ca9ddc43f0b0a8271ed202178d/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:07.806063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396e6d7-f794-4d77-ac9d-aee3280a76c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:07.810051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396e6d7-f794-4d77-ac9d-aee3280a76c9 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:07.817245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694a7c8-b7ca-4f46-ae38-d34f0f2d94e3 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:07.817635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694a7c8-b7ca-4f46-ae38-d34f0f2d94e3 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:07.817852 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694a7c8-b7ca-4f46-ae38-d34f0f2d94e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:07.848974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 789/1574] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:07.853100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3694a7c8-b7ca-4f46-ae38-d34f0f2d94e3 None None] MFA Rules not processed for user `2ef4a1ca9ddc43f0b0a8271ed202178d`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:07.853957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3694a7c8-b7ca-4f46-ae38-d34f0f2d94e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:07.856722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396e6d7-f794-4d77-ac9d-aee3280a76c9 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:07.874648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396e6d7-f794-4d77-ac9d-aee3280a76c9 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:07.946582 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 786/1575] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:07 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:07.951291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694a7c8-b7ca-4f46-ae38-d34f0f2d94e3 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:07.985060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc20408-9806-4797-8906-3b31ac0ea6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:07.985969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc20408-9806-4797-8906-3b31ac0ea6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:07.986432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc20408-9806-4797-8906-3b31ac0ea6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:07.986904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc20408-9806-4797-8906-3b31ac0ea6e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:07.987720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc20408-9806-4797-8906-3b31ac0ea6e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:07.990896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc20408-9806-4797-8906-3b31ac0ea6e1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.048596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 790/1576] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:07 2022] POST /identity/v3/projects => generated 397 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:41:08.048788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694a7c8-b7ca-4f46-ae38-d34f0f2d94e3 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.053019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc20408-9806-4797-8906-3b31ac0ea6e1 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.098191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:08.126893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc4558-4f63-4770-b66b-9c0698f75680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.127683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc4558-4f63-4770-b66b-9c0698f75680 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.127954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc4558-4f63-4770-b66b-9c0698f75680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.128286 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc4558-4f63-4770-b66b-9c0698f75680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.129017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc4558-4f63-4770-b66b-9c0698f75680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.131071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc4558-4f63-4770-b66b-9c0698f75680 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.138586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc20408-9806-4797-8906-3b31ac0ea6e1 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.143649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56af1c-975a-4b27-8b48-d314678ff950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.144300 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56af1c-975a-4b27-8b48-d314678ff950 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.144982 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56af1c-975a-4b27-8b48-d314678ff950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.144982 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56af1c-975a-4b27-8b48-d314678ff950 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.145623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56af1c-975a-4b27-8b48-d314678ff950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.147806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56af1c-975a-4b27-8b48-d314678ff950 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.173736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 787/1577] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:07 2022] POST /identity/v3/projects => generated 407 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:08.176882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 791/1578] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:08 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:08.178061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc4558-4f63-4770-b66b-9c0698f75680 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.181778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56af1c-975a-4b27-8b48-d314678ff950 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.196434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56af1c-975a-4b27-8b48-d314678ff950 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.199356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:08.252631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc4558-4f63-4770-b66b-9c0698f75680 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.257879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78cc3c-0d0f-4e54-89a9-c6b2e0f7ca17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.258501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78cc3c-0d0f-4e54-89a9-c6b2e0f7ca17 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.258814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78cc3c-0d0f-4e54-89a9-c6b2e0f7ca17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.259106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78cc3c-0d0f-4e54-89a9-c6b2e0f7ca17 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.259833 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78cc3c-0d0f-4e54-89a9-c6b2e0f7ca17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.262069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78cc3c-0d0f-4e54-89a9-c6b2e0f7ca17 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.275742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437dad-02a1-4396-99f8-a1b3c82c3094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.277013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437dad-02a1-4396-99f8-a1b3c82c3094 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.277293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437dad-02a1-4396-99f8-a1b3c82c3094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.277765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437dad-02a1-4396-99f8-a1b3c82c3094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.278481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9437dad-02a1-4396-99f8-a1b3c82c3094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.280218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9437dad-02a1-4396-99f8-a1b3c82c3094 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.288218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 788/1579] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:08 2022] POST /identity/v3/users => generated 310 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:08.288218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78cc3c-0d0f-4e54-89a9-c6b2e0f7ca17 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.288218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3d78cc3c-0d0f-4e54-89a9-c6b2e0f7ca17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:08.291863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 792/1580] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:41:08.293244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437dad-02a1-4396-99f8-a1b3c82c3094 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.300783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78cc3c-0d0f-4e54-89a9-c6b2e0f7ca17 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.303053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437dad-02a1-4396-99f8-a1b3c82c3094 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.305728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.306359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.306639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.306914 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] PATH_INFO: `/v3/users/8a8733f56d6d4a65b1af34a9b44c0ce9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.307916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d964a7-606a-4918-8a59-1ef9aea855a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.308084 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8733f56d6d4a65b1af34a9b44c0ce9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.309014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d964a7-606a-4918-8a59-1ef9aea855a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.309014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d964a7-606a-4918-8a59-1ef9aea855a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.309214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d964a7-606a-4918-8a59-1ef9aea855a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.309744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d964a7-606a-4918-8a59-1ef9aea855a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.310044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.311299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d964a7-606a-4918-8a59-1ef9aea855a0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.324089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 793/1581] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:08.325297 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d964a7-606a-4918-8a59-1ef9aea855a0 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.338246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d964a7-606a-4918-8a59-1ef9aea855a0 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.355606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4135881-5b16-4501-9d0f-9e1931cea36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.357552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4135881-5b16-4501-9d0f-9e1931cea36f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.357552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4135881-5b16-4501-9d0f-9e1931cea36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.357552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4135881-5b16-4501-9d0f-9e1931cea36f admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.357995 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4135881-5b16-4501-9d0f-9e1931cea36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.360137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4135881-5b16-4501-9d0f-9e1931cea36f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.372488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 794/1582] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:08.373377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4135881-5b16-4501-9d0f-9e1931cea36f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.389806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4135881-5b16-4501-9d0f-9e1931cea36f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.395429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8733f56d6d4a65b1af34a9b44c0ce9'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:08.396206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75620158-972c-40aa-bdb3-6071869c7f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.396409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75620158-972c-40aa-bdb3-6071869c7f51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.396677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75620158-972c-40aa-bdb3-6071869c7f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.396992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75620158-972c-40aa-bdb3-6071869c7f51 admin admin] PATH_INFO: `/v3/projects/04f7d5a87b214ff9a73cd31199e5e0d1/users/45afbfc71abd4e7a8180d612eca34ae5/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.417993 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75620158-972c-40aa-bdb3-6071869c7f51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f7d5a87b214ff9a73cd31199e5e0d1, user_id=45afbfc71abd4e7a8180d612eca34ae5, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.419963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8733f56d6d4a65b1af34a9b44c0ce9'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:08.421117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8733f56d6d4a65b1af34a9b44c0ce9'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:08.421775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75620158-972c-40aa-bdb3-6071869c7f51 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.446037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 795/1583] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:08 2022] PUT /identity/v3/projects/04f7d5a87b214ff9a73cd31199e5e0d1/users/45afbfc71abd4e7a8180d612eca34ae5/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:08.447560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75620158-972c-40aa-bdb3-6071869c7f51 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.450898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-75620158-972c-40aa-bdb3-6071869c7f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:08.454347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8733f56d6d4a65b1af34a9b44c0ce9'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:08.465387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8733f56d6d4a65b1af34a9b44c0ce9'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:08.475745 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 789/1584] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:08 2022] DELETE /identity/v3/users/8a8733f56d6d4a65b1af34a9b44c0ce9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:08.476354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.479208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:08.591900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75620158-972c-40aa-bdb3-6071869c7f51 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.601765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47984f-254c-425e-8cef-c650840a68ca admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.605786 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44a250-61a9-4e23-a002-66389897826d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.606512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44a250-61a9-4e23-a002-66389897826d admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.608458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44a250-61a9-4e23-a002-66389897826d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.608879 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44a250-61a9-4e23-a002-66389897826d admin admin] PATH_INFO: `/v3/projects/1c5402e165164bb8be81a299d823224f/users/50860191b1eb4f5db304b8d1e1aa9f80/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.609454 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84384a75-c7d8-465d-a295-36e577d1314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.610105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84384a75-c7d8-465d-a295-36e577d1314d admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.610552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84384a75-c7d8-465d-a295-36e577d1314d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.610830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84384a75-c7d8-465d-a295-36e577d1314d admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.611560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84384a75-c7d8-465d-a295-36e577d1314d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.613280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84384a75-c7d8-465d-a295-36e577d1314d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.626181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 790/1585] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:08.627883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe85d2-cda9-414e-a99d-89c6dd37f979 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.628174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe85d2-cda9-414e-a99d-89c6dd37f979 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.628895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe85d2-cda9-414e-a99d-89c6dd37f979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.638469 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:08.656256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44a250-61a9-4e23-a002-66389897826d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5402e165164bb8be81a299d823224f, user_id=50860191b1eb4f5db304b8d1e1aa9f80, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.657534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-89fe85d2-cda9-414e-a99d-89c6dd37f979 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:08.659274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-89fe85d2-cda9-414e-a99d-89c6dd37f979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:08.664611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44a250-61a9-4e23-a002-66389897826d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.674212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 791/1586] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:08.674938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe85d2-cda9-414e-a99d-89c6dd37f979 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.687478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe85d2-cda9-414e-a99d-89c6dd37f979 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.691921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:08.694424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 796/1587] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:08 2022] PUT /identity/v3/projects/1c5402e165164bb8be81a299d823224f/users/50860191b1eb4f5db304b8d1e1aa9f80/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:08.695657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44a250-61a9-4e23-a002-66389897826d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:08.701570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5a44a250-61a9-4e23-a002-66389897826d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:08.809953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966d398-ea7e-4ac6-b66b-d0cb48f45713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.811804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966d398-ea7e-4ac6-b66b-d0cb48f45713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.812101 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966d398-ea7e-4ac6-b66b-d0cb48f45713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.812391 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966d398-ea7e-4ac6-b66b-d0cb48f45713 admin admin] PATH_INFO: `/v3/projects/04f7d5a87b214ff9a73cd31199e5e0d1/users/45afbfc71abd4e7a8180d612eca34ae5/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.819198 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966d398-ea7e-4ac6-b66b-d0cb48f45713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f7d5a87b214ff9a73cd31199e5e0d1, user_id=45afbfc71abd4e7a8180d612eca34ae5, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:08.823312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966d398-ea7e-4ac6-b66b-d0cb48f45713 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:08.834749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44a250-61a9-4e23-a002-66389897826d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:08.864326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 792/1588] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:08 2022] PUT /identity/v3/projects/04f7d5a87b214ff9a73cd31199e5e0d1/users/45afbfc71abd4e7a8180d612eca34ae5/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:08.866520 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bdf00-fa95-460e-a4d2-42f62ab0943a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.866869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bdf00-fa95-460e-a4d2-42f62ab0943a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.867287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bdf00-fa95-460e-a4d2-42f62ab0943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:08.908592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e76bdf00-fa95-460e-a4d2-42f62ab0943a None None] MFA Rules not processed for user `50860191b1eb4f5db304b8d1e1aa9f80`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:08.909991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e76bdf00-fa95-460e-a4d2-42f62ab0943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:08.991368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba6b74-e4ed-4172-b6a8-27801ae49bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:08.992618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba6b74-e4ed-4172-b6a8-27801ae49bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:08.993113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba6b74-e4ed-4172-b6a8-27801ae49bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:08.993398 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba6b74-e4ed-4172-b6a8-27801ae49bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.048810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 793/1589] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:08 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:09.051180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110a5ed-e7c4-4639-ae82-fb70518d5524 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.051472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110a5ed-e7c4-4639-ae82-fb70518d5524 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.053235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110a5ed-e7c4-4639-ae82-fb70518d5524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.096028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba6b74-e4ed-4172-b6a8-27801ae49bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.099108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba6b74-e4ed-4172-b6a8-27801ae49bf5 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.119997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d110a5ed-e7c4-4639-ae82-fb70518d5524 None None] MFA Rules not processed for user `45afbfc71abd4e7a8180d612eca34ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:09.122983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d110a5ed-e7c4-4639-ae82-fb70518d5524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:09.125098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 797/1590] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:09.126325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba6b74-e4ed-4172-b6a8-27801ae49bf5 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.140145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba6b74-e4ed-4172-b6a8-27801ae49bf5 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.145833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a03215-08d7-461a-a2f6-5ded69f3280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.146746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a03215-08d7-461a-a2f6-5ded69f3280d admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.147125 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a03215-08d7-461a-a2f6-5ded69f3280d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.147504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a03215-08d7-461a-a2f6-5ded69f3280d admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.148420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a03215-08d7-461a-a2f6-5ded69f3280d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.150895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a03215-08d7-461a-a2f6-5ded69f3280d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.201318 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 798/1591] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:09 2022] POST /identity/v3/projects => generated 401 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:09.204195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a03215-08d7-461a-a2f6-5ded69f3280d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.262140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 794/1592] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:09 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:09.275509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110a5ed-e7c4-4639-ae82-fb70518d5524 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.275667 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:09.278345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d110a5ed-e7c4-4639-ae82-fb70518d5524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:09.301316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:09.333483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a03215-08d7-461a-a2f6-5ded69f3280d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.341726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210620d-db0b-4b84-a098-1d2931900f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.342692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210620d-db0b-4b84-a098-1d2931900f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.343150 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210620d-db0b-4b84-a098-1d2931900f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.343599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210620d-db0b-4b84-a098-1d2931900f39 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.344934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210620d-db0b-4b84-a098-1d2931900f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.349914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210620d-db0b-4b84-a098-1d2931900f39 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.379841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110a5ed-e7c4-4639-ae82-fb70518d5524 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.404887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 799/1593] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:09 2022] POST /identity/v3/users => generated 307 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:09.412288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210620d-db0b-4b84-a098-1d2931900f39 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.437294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210620d-db0b-4b84-a098-1d2931900f39 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.445641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c831b-c7d4-44cf-8762-a51ae5513e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.446843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c831b-c7d4-44cf-8762-a51ae5513e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.447380 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c831b-c7d4-44cf-8762-a51ae5513e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.447894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c831b-c7d4-44cf-8762-a51ae5513e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.449596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c831b-c7d4-44cf-8762-a51ae5513e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.452880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c831b-c7d4-44cf-8762-a51ae5513e37 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.476971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 800/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:09.479211 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bdf4d-802f-4579-b270-287d44eee5af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.481179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bdf4d-802f-4579-b270-287d44eee5af service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.482030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c831b-c7d4-44cf-8762-a51ae5513e37 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.482404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bdf4d-802f-4579-b270-287d44eee5af service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.482809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bdf4d-802f-4579-b270-287d44eee5af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.494707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c831b-c7d4-44cf-8762-a51ae5513e37 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.500529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f8702-8a2f-4584-9567-7d176ae19236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.501233 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8702-8a2f-4584-9567-7d176ae19236 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.501514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8702-8a2f-4584-9567-7d176ae19236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.501788 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8702-8a2f-4584-9567-7d176ae19236 admin admin] PATH_INFO: `/v3/projects/7b867f58452d41a2b35fbaf89c445e7e/users/946f0e32d908445c90e1b22c51e27135/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.519566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f8702-8a2f-4584-9567-7d176ae19236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b867f58452d41a2b35fbaf89c445e7e, user_id=946f0e32d908445c90e1b22c51e27135, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.522289 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f8702-8a2f-4584-9567-7d176ae19236 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.536899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 801/1595] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:09 2022] PUT /identity/v3/projects/7b867f58452d41a2b35fbaf89c445e7e/users/946f0e32d908445c90e1b22c51e27135/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:09.540737 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f8702-8a2f-4584-9567-7d176ae19236 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.561875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bdf4d-802f-4579-b270-287d44eee5af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.563395 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bdf4d-802f-4579-b270-287d44eee5af service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.575476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 795/1596] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:09 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:09.584241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bdf4d-802f-4579-b270-287d44eee5af service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.585855 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-446bdf4d-802f-4579-b270-287d44eee5af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:09.619259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bdf4d-802f-4579-b270-287d44eee5af service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.640632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f8702-8a2f-4584-9567-7d176ae19236 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.646319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657360b-6ac7-4e2c-9490-3507e2271e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.646950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657360b-6ac7-4e2c-9490-3507e2271e3f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.647396 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657360b-6ac7-4e2c-9490-3507e2271e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.647667 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657360b-6ac7-4e2c-9490-3507e2271e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.648316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657360b-6ac7-4e2c-9490-3507e2271e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.649969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657360b-6ac7-4e2c-9490-3507e2271e3f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.661161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 802/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:09.665944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657360b-6ac7-4e2c-9490-3507e2271e3f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.676388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657360b-6ac7-4e2c-9490-3507e2271e3f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.681485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddddae-5c4b-4302-9590-dbc137a6d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.682162 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddddae-5c4b-4302-9590-dbc137a6d14a admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.683059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddddae-5c4b-4302-9590-dbc137a6d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.683239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddddae-5c4b-4302-9590-dbc137a6d14a admin admin] PATH_INFO: `/v3/projects/7b867f58452d41a2b35fbaf89c445e7e/users/946f0e32d908445c90e1b22c51e27135/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.685286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddddae-5c4b-4302-9590-dbc137a6d14a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b867f58452d41a2b35fbaf89c445e7e, user_id=946f0e32d908445c90e1b22c51e27135, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.688145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddddae-5c4b-4302-9590-dbc137a6d14a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.710729 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.711555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 803/1598] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:09 2022] PUT /identity/v3/projects/7b867f58452d41a2b35fbaf89c445e7e/users/946f0e32d908445c90e1b22c51e27135/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:09.711805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.712015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.712808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] PATH_INFO: `/v3/projects/2a43dd88e77c4a66b30a2e63e5e7103a` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.712978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f093c-74ba-4b5e-adf4-d9f9383689ba None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.713123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f093c-74ba-4b5e-adf4-d9f9383689ba None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.713379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f093c-74ba-4b5e-adf4-d9f9383689ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.713740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a43dd88e77c4a66b30a2e63e5e7103a)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.716211 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.752646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:09.761481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-bf6f093c-74ba-4b5e-adf4-d9f9383689ba None None] MFA Rules not processed for user `946f0e32d908445c90e1b22c51e27135`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:09.763049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-bf6f093c-74ba-4b5e-adf4-d9f9383689ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:09.781166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a43dd88e77c4a66b30a2e63e5e7103a', 'request_id': 'req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:09.788997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a43dd88e77c4a66b30a2e63e5e7103a', 'request_id': 'req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:09.798983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a43dd88e77c4a66b30a2e63e5e7103a', 'request_id': 'req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:09.814617 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] The token cache is being invalidate because project 2a43dd88e77c4a66b30a2e63e5e7103a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:09.815247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:09.815925 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:09.817952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 796/1599] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:09 2022] DELETE /identity/v3/projects/2a43dd88e77c4a66b30a2e63e5e7103a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:09.818685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.820043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:09.840110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:09.847761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 804/1600] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:09 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:09.848635 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f093c-74ba-4b5e-adf4-d9f9383689ba None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.850554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-bf6f093c-74ba-4b5e-adf4-d9f9383689ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:09.888207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0d5ca-2874-4e0b-b26c-2db6cbafb7c3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.891885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0955526-2ab2-4eba-855a-c63e2431da33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.892494 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0955526-2ab2-4eba-855a-c63e2431da33 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.892767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0955526-2ab2-4eba-855a-c63e2431da33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.893068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0955526-2ab2-4eba-855a-c63e2431da33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.894981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b0955526-2ab2-4eba-855a-c63e2431da33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:09.923683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f093c-74ba-4b5e-adf4-d9f9383689ba None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.964365 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0955526-2ab2-4eba-855a-c63e2431da33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.966197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0955526-2ab2-4eba-855a-c63e2431da33 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.976311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 797/1601] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:09 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:09.977219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0955526-2ab2-4eba-855a-c63e2431da33 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:09.978518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b0955526-2ab2-4eba-855a-c63e2431da33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:09.988718 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.989420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.989697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.989970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] PATH_INFO: `/v3/users/72db2349f3a7489eb4242f7a35df7a44` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:09.990403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0955526-2ab2-4eba-855a-c63e2431da33 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:09.991171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72db2349f3a7489eb4242f7a35df7a44)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:09.993469 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:09.995541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59eec0-6db0-4229-9443-16e796a1e185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:09.996299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59eec0-6db0-4229-9443-16e796a1e185 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:09.996660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59eec0-6db0-4229-9443-16e796a1e185 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:09.997117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59eec0-6db0-4229-9443-16e796a1e185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:10.086155 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72db2349f3a7489eb4242f7a35df7a44'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:10.123077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72db2349f3a7489eb4242f7a35df7a44'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:10.123920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72db2349f3a7489eb4242f7a35df7a44'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:10.150394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72db2349f3a7489eb4242f7a35df7a44'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:10.151865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72db2349f3a7489eb4242f7a35df7a44'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:10.174012 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59eec0-6db0-4229-9443-16e796a1e185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:10.177005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59eec0-6db0-4229-9443-16e796a1e185 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:10.190312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:10.195440 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 805/1602] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:09 2022] DELETE /identity/v3/users/72db2349f3a7489eb4242f7a35df7a44 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:10.199872 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 798/1603] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:09 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:10.562957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:10.564760 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:10.631495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:10.714637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd7c87-5850-444f-aa27-2ddbb0168594 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:10.723538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e320b7-7f18-4b45-bdec-2225c7cabf5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:10.724178 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e320b7-7f18-4b45-bdec-2225c7cabf5d service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:10.724460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e320b7-7f18-4b45-bdec-2225c7cabf5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:10.724728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e320b7-7f18-4b45-bdec-2225c7cabf5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:10.726736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-89e320b7-7f18-4b45-bdec-2225c7cabf5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:10.846032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e320b7-7f18-4b45-bdec-2225c7cabf5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:10.846841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59eec0-6db0-4229-9443-16e796a1e185 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:10.849253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e320b7-7f18-4b45-bdec-2225c7cabf5d service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:10.861227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59eec0-6db0-4229-9443-16e796a1e185 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:10.867337 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 806/1604] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:10 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:10.965710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:10.966631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:10.966907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:10.967183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] PATH_INFO: `/v3/projects/216b9a0f4e3f4efaad8cb3d4b1d3be92` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:10.968419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=216b9a0f4e3f4efaad8cb3d4b1d3be92)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:10.970541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:11.045854 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '216b9a0f4e3f4efaad8cb3d4b1d3be92', 'request_id': 'req-9200e6dd-da36-41ef-9622-d0a067b3eb9f'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:11.055252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '216b9a0f4e3f4efaad8cb3d4b1d3be92', 'request_id': 'req-9200e6dd-da36-41ef-9622-d0a067b3eb9f'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:11.065220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '216b9a0f4e3f4efaad8cb3d4b1d3be92', 'request_id': 'req-9200e6dd-da36-41ef-9622-d0a067b3eb9f'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:11.083833 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] The token cache is being invalidate because project 216b9a0f4e3f4efaad8cb3d4b1d3be92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:11.084461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:11.085808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:11.088175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 799/1605] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:10 2022] DELETE /identity/v3/projects/216b9a0f4e3f4efaad8cb3d4b1d3be92 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:11.092555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e320b7-7f18-4b45-bdec-2225c7cabf5d service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:11.094200 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-89e320b7-7f18-4b45-bdec-2225c7cabf5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:11.108703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e320b7-7f18-4b45-bdec-2225c7cabf5d service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:11.213393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:11.216193 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:11.216599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:11.218051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] PATH_INFO: `/v3/users/1c49851b2d964a91b5641b28a0a760ac` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:11.218913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c49851b2d964a91b5641b28a0a760ac)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:11.223050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:11.344109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c49851b2d964a91b5641b28a0a760ac'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:11.387870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c49851b2d964a91b5641b28a0a760ac'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:11.391729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c49851b2d964a91b5641b28a0a760ac'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:11.418115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c49851b2d964a91b5641b28a0a760ac'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:11.420354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d965893c-9da6-4750-af93-062ca11e3e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c49851b2d964a91b5641b28a0a760ac'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:11.448678 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:11.476798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 807/1606] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:11 2022] DELETE /identity/v3/users/1c49851b2d964a91b5641b28a0a760ac => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:12.466646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:12.468111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:12.509494 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200e6dd-da36-41ef-9622-d0a067b3eb9f admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:12.583049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:12.583957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:12.584241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:12.584517 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] PATH_INFO: `/v3/projects/99dd2a2d63104f6ab51c1eb20f2bf81a` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:12.585785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99dd2a2d63104f6ab51c1eb20f2bf81a)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:12.587653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:12.633031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:12.660929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99dd2a2d63104f6ab51c1eb20f2bf81a', 'request_id': 'req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:12.672644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99dd2a2d63104f6ab51c1eb20f2bf81a', 'request_id': 'req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:12.686693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99dd2a2d63104f6ab51c1eb20f2bf81a', 'request_id': 'req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:12.711969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] The token cache is being invalidate because project 99dd2a2d63104f6ab51c1eb20f2bf81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:12.712368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:12.714105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-03d836e3-6ec1-4e5d-9b0b-0f3c1a5f3494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:12.717911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 800/1607] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:12 2022] DELETE /identity/v3/projects/99dd2a2d63104f6ab51c1eb20f2bf81a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:12.724042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802e1dd-91de-408f-b132-f0b9a1ecdd31 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:12.725077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802e1dd-91de-408f-b132-f0b9a1ecdd31 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:12.725671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802e1dd-91de-408f-b132-f0b9a1ecdd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:12.748046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-8802e1dd-91de-408f-b132-f0b9a1ecdd31 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:12.749288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8802e1dd-91de-408f-b132-f0b9a1ecdd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:12.856103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 808/1608] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:12.866771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b586a-a3dc-451f-bf18-2ee93ecb0657 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:12.868090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b586a-a3dc-451f-bf18-2ee93ecb0657 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:12.868570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b586a-a3dc-451f-bf18-2ee93ecb0657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:12.896150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-952b586a-a3dc-451f-bf18-2ee93ecb0657 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:12.897803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-952b586a-a3dc-451f-bf18-2ee93ecb0657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:12.908716 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 801/1609] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:12.913938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802e1dd-91de-408f-b132-f0b9a1ecdd31 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:12.926636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802e1dd-91de-408f-b132-f0b9a1ecdd31 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:12.932733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a80097-8f70-4439-93ef-79770aedb582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:12.933630 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a80097-8f70-4439-93ef-79770aedb582 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:12.933975 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a80097-8f70-4439-93ef-79770aedb582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:12.934407 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a80097-8f70-4439-93ef-79770aedb582 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:12.935222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a80097-8f70-4439-93ef-79770aedb582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:12.935870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a80097-8f70-4439-93ef-79770aedb582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:12.937520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a80097-8f70-4439-93ef-79770aedb582 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:12.950642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 809/1610] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:12.955037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b586a-a3dc-451f-bf18-2ee93ecb0657 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:12.968083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b586a-a3dc-451f-bf18-2ee93ecb0657 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:12.973576 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5f521-4115-4495-9073-c689b8c22677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:12.974199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5f521-4115-4495-9073-c689b8c22677 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:12.974476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5f521-4115-4495-9073-c689b8c22677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:12.974742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5f521-4115-4495-9073-c689b8c22677 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:12.975419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5f521-4115-4495-9073-c689b8c22677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:12.977816 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5f521-4115-4495-9073-c689b8c22677 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.025682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 802/1611] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:12 2022] POST /identity/v3/projects => generated 391 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:41:13.027472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a80097-8f70-4439-93ef-79770aedb582 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:13.039877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a80097-8f70-4439-93ef-79770aedb582 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:13.110258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ecbe6-42b3-49dd-b8e3-563ff9f202a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:13.111083 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ecbe6-42b3-49dd-b8e3-563ff9f202a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.111452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ecbe6-42b3-49dd-b8e3-563ff9f202a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.111791 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ecbe6-42b3-49dd-b8e3-563ff9f202a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.112700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ecbe6-42b3-49dd-b8e3-563ff9f202a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:13.114944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ecbe6-42b3-49dd-b8e3-563ff9f202a8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.140950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 810/1612] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:13 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:13.142453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5f521-4115-4495-9073-c689b8c22677 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:13.154406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5f521-4115-4495-9073-c689b8c22677 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:13.159898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38c44e-9cc8-431b-92f8-00cce5acd65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:13.160636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38c44e-9cc8-431b-92f8-00cce5acd65d admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.161099 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38c44e-9cc8-431b-92f8-00cce5acd65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.161434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38c44e-9cc8-431b-92f8-00cce5acd65d admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.162368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38c44e-9cc8-431b-92f8-00cce5acd65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:13.164129 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38c44e-9cc8-431b-92f8-00cce5acd65d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.180965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 803/1613] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:13.182457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ecbe6-42b3-49dd-b8e3-563ff9f202a8 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:13.193195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ecbe6-42b3-49dd-b8e3-563ff9f202a8 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:13.198555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773eaaa-27a5-433b-817d-44f47356c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:13.199155 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773eaaa-27a5-433b-817d-44f47356c23f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.199428 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773eaaa-27a5-433b-817d-44f47356c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.199694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773eaaa-27a5-433b-817d-44f47356c23f admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.200323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773eaaa-27a5-433b-817d-44f47356c23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:13.201937 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773eaaa-27a5-433b-817d-44f47356c23f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.213354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 811/1614] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:13.218099 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38c44e-9cc8-431b-92f8-00cce5acd65d admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:13.231188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38c44e-9cc8-431b-92f8-00cce5acd65d admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:13.237588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667bd00-e8de-430f-b456-997c9ed8193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:13.239387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667bd00-e8de-430f-b456-997c9ed8193c admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.239663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667bd00-e8de-430f-b456-997c9ed8193c admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.240017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667bd00-e8de-430f-b456-997c9ed8193c admin admin] PATH_INFO: `/v3/projects/01df70b705804ceca3a2e8cde5dd9372/users/bc7e1918774c4bbf9cbc83db40b0c85d/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.262225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667bd00-e8de-430f-b456-997c9ed8193c admin admin] RBAC: Authorizing `identity:create_grant(project_id=01df70b705804ceca3a2e8cde5dd9372, user_id=bc7e1918774c4bbf9cbc83db40b0c85d, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:13.265051 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667bd00-e8de-430f-b456-997c9ed8193c admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.284447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 804/1615] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:13 2022] PUT /identity/v3/projects/01df70b705804ceca3a2e8cde5dd9372/users/bc7e1918774c4bbf9cbc83db40b0c85d/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:13.298985 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0cdd7-6ab6-40a7-9b5e-66818ca15b3c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.299333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0cdd7-6ab6-40a7-9b5e-66818ca15b3c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.299592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0cdd7-6ab6-40a7-9b5e-66818ca15b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.330545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-27b0cdd7-6ab6-40a7-9b5e-66818ca15b3c None None] MFA Rules not processed for user `bc7e1918774c4bbf9cbc83db40b0c85d`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:13.331775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-27b0cdd7-6ab6-40a7-9b5e-66818ca15b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:13.397416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:13.409146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 812/1616] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:13.413625 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667bd00-e8de-430f-b456-997c9ed8193c admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:13.477085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667bd00-e8de-430f-b456-997c9ed8193c admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:13.482085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562299b6-caa1-48c2-9b95-ba366e1296d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:13.482712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562299b6-caa1-48c2-9b95-ba366e1296d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.482990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562299b6-caa1-48c2-9b95-ba366e1296d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.483254 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562299b6-caa1-48c2-9b95-ba366e1296d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.483959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562299b6-caa1-48c2-9b95-ba366e1296d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:13.489936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562299b6-caa1-48c2-9b95-ba366e1296d9 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.522727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 805/1617] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:13 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:13.526881 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0cdd7-6ab6-40a7-9b5e-66818ca15b3c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:13.543095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0cdd7-6ab6-40a7-9b5e-66818ca15b3c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:13.601954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17404bd-b2b2-48d3-ba99-23c975fcf999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:13.602704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17404bd-b2b2-48d3-ba99-23c975fcf999 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.602990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17404bd-b2b2-48d3-ba99-23c975fcf999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.603261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17404bd-b2b2-48d3-ba99-23c975fcf999 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.603895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17404bd-b2b2-48d3-ba99-23c975fcf999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:13.606430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17404bd-b2b2-48d3-ba99-23c975fcf999 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.629640 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 813/1618] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:13 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:41:13.633480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562299b6-caa1-48c2-9b95-ba366e1296d9 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:13.644166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562299b6-caa1-48c2-9b95-ba366e1296d9 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:13.648887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3236f-a742-410e-932b-fb203daf4f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:13.649507 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3236f-a742-410e-932b-fb203daf4f30 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.649782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3236f-a742-410e-932b-fb203daf4f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.650049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3236f-a742-410e-932b-fb203daf4f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.650690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3236f-a742-410e-932b-fb203daf4f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:13.654163 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3236f-a742-410e-932b-fb203daf4f30 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.666154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 806/1619] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:13.670808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17404bd-b2b2-48d3-ba99-23c975fcf999 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:13.681783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17404bd-b2b2-48d3-ba99-23c975fcf999 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:13.688098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb3550-f78e-4233-ad32-c90ef564eb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:13.688768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb3550-f78e-4233-ad32-c90ef564eb8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.689082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb3550-f78e-4233-ad32-c90ef564eb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.689359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb3550-f78e-4233-ad32-c90ef564eb8f admin admin] PATH_INFO: `/v3/projects/60b0a7c2f491416a9694f7637d8211e2/users/0be4b914da094e2d9f0f1b12f3c1db97/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.706554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb3550-f78e-4233-ad32-c90ef564eb8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b0a7c2f491416a9694f7637d8211e2, user_id=0be4b914da094e2d9f0f1b12f3c1db97, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:13.709253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb3550-f78e-4233-ad32-c90ef564eb8f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.723745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 814/1620] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:13 2022] PUT /identity/v3/projects/60b0a7c2f491416a9694f7637d8211e2/users/0be4b914da094e2d9f0f1b12f3c1db97/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:13.727714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3236f-a742-410e-932b-fb203daf4f30 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:13.738761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3236f-a742-410e-932b-fb203daf4f30 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:13.769626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:13.801300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aa4b8-d1fc-493d-9dbb-6bba39c1410b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:13.802081 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aa4b8-d1fc-493d-9dbb-6bba39c1410b admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.802367 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aa4b8-d1fc-493d-9dbb-6bba39c1410b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.802705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aa4b8-d1fc-493d-9dbb-6bba39c1410b admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.803470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0aa4b8-d1fc-493d-9dbb-6bba39c1410b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:13.805337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0aa4b8-d1fc-493d-9dbb-6bba39c1410b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.817038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 807/1621] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:13.821424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb3550-f78e-4233-ad32-c90ef564eb8f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:13.832272 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb3550-f78e-4233-ad32-c90ef564eb8f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:13.837480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b0f50-592b-4402-8fca-72b31e1d8a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:13.838134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b0f50-592b-4402-8fca-72b31e1d8a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.838413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b0f50-592b-4402-8fca-72b31e1d8a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.838677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b0f50-592b-4402-8fca-72b31e1d8a24 admin admin] PATH_INFO: `/v3/projects/60b0a7c2f491416a9694f7637d8211e2/users/0be4b914da094e2d9f0f1b12f3c1db97/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.840548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b0f50-592b-4402-8fca-72b31e1d8a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b0a7c2f491416a9694f7637d8211e2, user_id=0be4b914da094e2d9f0f1b12f3c1db97, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:13.843483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b0f50-592b-4402-8fca-72b31e1d8a24 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:13.855224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 815/1622] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:13 2022] PUT /identity/v3/projects/60b0a7c2f491416a9694f7637d8211e2/users/0be4b914da094e2d9f0f1b12f3c1db97/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:13.868309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570a71d-c4ba-444b-9332-31b951068b21 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:13.868593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570a71d-c4ba-444b-9332-31b951068b21 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:13.868898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570a71d-c4ba-444b-9332-31b951068b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:13.904128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-8570a71d-c4ba-444b-9332-31b951068b21 None None] MFA Rules not processed for user `0be4b914da094e2d9f0f1b12f3c1db97`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:13.905452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8570a71d-c4ba-444b-9332-31b951068b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:13.989783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 808/1623] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:13 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:14.004079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b0f50-592b-4402-8fca-72b31e1d8a24 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:14.004463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3b0b0f50-592b-4402-8fca-72b31e1d8a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:14.079501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b0f50-592b-4402-8fca-72b31e1d8a24 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:14.083483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369d3f4-365f-44ef-aafd-92422b99f63f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:14.084211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369d3f4-365f-44ef-aafd-92422b99f63f service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:14.084589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369d3f4-365f-44ef-aafd-92422b99f63f service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:14.084998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369d3f4-365f-44ef-aafd-92422b99f63f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:14.096028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369d3f4-365f-44ef-aafd-92422b99f63f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:14.097634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369d3f4-365f-44ef-aafd-92422b99f63f service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:14.109151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 816/1624] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:14 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:14.171753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570a71d-c4ba-444b-9332-31b951068b21 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:14.183986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570a71d-c4ba-444b-9332-31b951068b21 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:14.187630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e803e15-9320-4e8a-8c91-a89db6058e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:14.188233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e803e15-9320-4e8a-8c91-a89db6058e5e service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:14.188509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e803e15-9320-4e8a-8c91-a89db6058e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:14.188780 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e803e15-9320-4e8a-8c91-a89db6058e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:14.253206 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e803e15-9320-4e8a-8c91-a89db6058e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:14.254708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e803e15-9320-4e8a-8c91-a89db6058e5e service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:14.264177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 809/1625] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:16.384337 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369d3f4-365f-44ef-aafd-92422b99f63f service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:16.385893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7369d3f4-365f-44ef-aafd-92422b99f63f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:16.469082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:16.488258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369d3f4-365f-44ef-aafd-92422b99f63f service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:16.494750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:16.495570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:16.495815 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:16.496018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] PATH_INFO: `/v3/users/50860191b1eb4f5db304b8d1e1aa9f80` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:16.497773 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50860191b1eb4f5db304b8d1e1aa9f80)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:16.502585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:16.585625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50860191b1eb4f5db304b8d1e1aa9f80'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:16.606933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50860191b1eb4f5db304b8d1e1aa9f80'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:16.608038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50860191b1eb4f5db304b8d1e1aa9f80'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:16.620312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50860191b1eb4f5db304b8d1e1aa9f80'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:16.621252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50860191b1eb4f5db304b8d1e1aa9f80'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:16.643508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 817/1626] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:16 2022] DELETE /identity/v3/users/50860191b1eb4f5db304b8d1e1aa9f80 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:16.648299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a9b0-71d6-4386-8095-07a099c542a2 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:16.648741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a9b0-71d6-4386-8095-07a099c542a2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:16.649188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a9b0-71d6-4386-8095-07a099c542a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:16.678053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4681a9b0-71d6-4386-8095-07a099c542a2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:16.679300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4681a9b0-71d6-4386-8095-07a099c542a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:16.763668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:16.790607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 810/1627] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:16.802014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:16.805055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:16.896534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446485d8-59ae-4afa-a544-eb7ae408c720 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:16.900820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b15975-7291-483b-a952-01be9ce2792e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:16.901485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b15975-7291-483b-a952-01be9ce2792e service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:16.901764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b15975-7291-483b-a952-01be9ce2792e service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:16.902035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b15975-7291-483b-a952-01be9ce2792e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:16.918401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b15975-7291-483b-a952-01be9ce2792e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:16.920280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b15975-7291-483b-a952-01be9ce2792e service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:16.949659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 818/1628] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:17.624215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681a9b0-71d6-4386-8095-07a099c542a2 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:17.625928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4681a9b0-71d6-4386-8095-07a099c542a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:17.639153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681a9b0-71d6-4386-8095-07a099c542a2 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:17.644189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:17.644832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:17.645145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:17.645417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] PATH_INFO: `/v3/projects/1c5402e165164bb8be81a299d823224f` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:17.646563 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5402e165164bb8be81a299d823224f)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:17.648477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:17.703981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5402e165164bb8be81a299d823224f', 'request_id': 'req-6ee2c709-ace4-41c0-848c-21a890721d68'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:17.710778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5402e165164bb8be81a299d823224f', 'request_id': 'req-6ee2c709-ace4-41c0-848c-21a890721d68'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:17.721184 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5402e165164bb8be81a299d823224f', 'request_id': 'req-6ee2c709-ace4-41c0-848c-21a890721d68'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:17.736578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] The token cache is being invalidate because project 1c5402e165164bb8be81a299d823224f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:17.737006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:17.737749 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:17.740472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 811/1629] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:17 2022] DELETE /identity/v3/projects/1c5402e165164bb8be81a299d823224f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:17.745708 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b15975-7291-483b-a952-01be9ce2792e service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:17.761295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b15975-7291-483b-a952-01be9ce2792e service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:17.833636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:17.834353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:17.834619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:17.835199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] PATH_INFO: `/v3/users/946f0e32d908445c90e1b22c51e27135` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:17.839479 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=946f0e32d908445c90e1b22c51e27135)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:17.841675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:17.877965 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:17.931863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '946f0e32d908445c90e1b22c51e27135'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:17.955234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '946f0e32d908445c90e1b22c51e27135'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:17.956273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '946f0e32d908445c90e1b22c51e27135'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:17.966033 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '946f0e32d908445c90e1b22c51e27135'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:17.966668 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2a671c3b-cdc8-426f-bd48-e3da9dfc1a8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '946f0e32d908445c90e1b22c51e27135'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:17.985447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 819/1630] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:17 2022] DELETE /identity/v3/users/946f0e32d908445c90e1b22c51e27135 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:18.507767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:18.509559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:18.532078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2c709-ace4-41c0-848c-21a890721d68 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:18.644997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:18.647559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:18.648066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:18.650742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] PATH_INFO: `/v3/projects/7b867f58452d41a2b35fbaf89c445e7e` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:18.652440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b867f58452d41a2b35fbaf89c445e7e)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:18.657426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:18.769794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b867f58452d41a2b35fbaf89c445e7e', 'request_id': 'req-5bc8d608-d700-4c3d-b94a-97655549ecee'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:18.777673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b867f58452d41a2b35fbaf89c445e7e', 'request_id': 'req-5bc8d608-d700-4c3d-b94a-97655549ecee'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:18.788106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b867f58452d41a2b35fbaf89c445e7e', 'request_id': 'req-5bc8d608-d700-4c3d-b94a-97655549ecee'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:18.802945 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] The token cache is being invalidate because project 7b867f58452d41a2b35fbaf89c445e7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:18.803307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:18.812607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5bc8d608-d700-4c3d-b94a-97655549ecee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:18.816126 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 812/1631] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:18 2022] DELETE /identity/v3/projects/7b867f58452d41a2b35fbaf89c445e7e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:18.821217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349a1a6-4983-4c9c-abb2-2ff9c8b56108 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:18.821927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349a1a6-4983-4c9c-abb2-2ff9c8b56108 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:18.821927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349a1a6-4983-4c9c-abb2-2ff9c8b56108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:18.843225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-f349a1a6-4983-4c9c-abb2-2ff9c8b56108 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:18.844455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f349a1a6-4983-4c9c-abb2-2ff9c8b56108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:18.926373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1eb19f-0a33-4118-90fa-ebe33b11c930 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:18.926662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1eb19f-0a33-4118-90fa-ebe33b11c930 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:18.926942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1eb19f-0a33-4118-90fa-ebe33b11c930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:18.931470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 820/1632] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:18.940978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:18.959943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9c1eb19f-0a33-4118-90fa-ebe33b11c930 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:18.963715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9c1eb19f-0a33-4118-90fa-ebe33b11c930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:18.981325 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 813/1633] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:18.983310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349a1a6-4983-4c9c-abb2-2ff9c8b56108 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:18.998645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349a1a6-4983-4c9c-abb2-2ff9c8b56108 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:19.009649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e6e3f-38ce-4407-a780-e1cf77e9134a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:19.012306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e6e3f-38ce-4407-a780-e1cf77e9134a admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.014207 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e6e3f-38ce-4407-a780-e1cf77e9134a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.014829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e6e3f-38ce-4407-a780-e1cf77e9134a admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.017390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e6e3f-38ce-4407-a780-e1cf77e9134a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:19.018545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e6e3f-38ce-4407-a780-e1cf77e9134a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:19.022038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e6e3f-38ce-4407-a780-e1cf77e9134a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:19.051011 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 821/1634] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:19.055837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1eb19f-0a33-4118-90fa-ebe33b11c930 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:19.067730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1eb19f-0a33-4118-90fa-ebe33b11c930 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:19.072988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd27fb-2a61-453f-b843-b393bd2a5029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:19.073645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd27fb-2a61-453f-b843-b393bd2a5029 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.073949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd27fb-2a61-453f-b843-b393bd2a5029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.074237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd27fb-2a61-453f-b843-b393bd2a5029 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.074892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd27fb-2a61-453f-b843-b393bd2a5029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:19.092326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd27fb-2a61-453f-b843-b393bd2a5029 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:19.156496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 814/1635] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:19 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:19.163681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e6e3f-38ce-4407-a780-e1cf77e9134a admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:19.197618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e6e3f-38ce-4407-a780-e1cf77e9134a admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:19.271425 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd27fb-2a61-453f-b843-b393bd2a5029 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:19.273878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-88dd27fb-2a61-453f-b843-b393bd2a5029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:19.304696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abc861-2457-45a4-806c-c7428454cf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:19.305505 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abc861-2457-45a4-806c-c7428454cf72 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.305882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abc861-2457-45a4-806c-c7428454cf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.306251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27abc861-2457-45a4-806c-c7428454cf72 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.307067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27abc861-2457-45a4-806c-c7428454cf72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:19.309192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27abc861-2457-45a4-806c-c7428454cf72 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:19.336274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 822/1636] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:19 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:19.340376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abc861-2457-45a4-806c-c7428454cf72 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:19.365683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27abc861-2457-45a4-806c-c7428454cf72 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:19.374663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49815026-d116-4d5a-9994-453de06a7fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:19.375589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49815026-d116-4d5a-9994-453de06a7fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.376058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49815026-d116-4d5a-9994-453de06a7fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.376346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49815026-d116-4d5a-9994-453de06a7fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.377470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49815026-d116-4d5a-9994-453de06a7fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:19.379383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49815026-d116-4d5a-9994-453de06a7fa8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:19.381078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:19.384166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd27fb-2a61-453f-b843-b393bd2a5029 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:19.393555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:19.394385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.394695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.394982 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] PATH_INFO: `/v3/users/bc7e1918774c4bbf9cbc83db40b0c85d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.396614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7e1918774c4bbf9cbc83db40b0c85d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:19.399962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 823/1637] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:41:19.400158 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:19.401777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49815026-d116-4d5a-9994-453de06a7fa8 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:19.421338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49815026-d116-4d5a-9994-453de06a7fa8 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:19.430912 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adadf3a-89ea-4046-bee4-06e548243c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:19.432137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adadf3a-89ea-4046-bee4-06e548243c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.433620 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adadf3a-89ea-4046-bee4-06e548243c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.434032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adadf3a-89ea-4046-bee4-06e548243c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.435425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adadf3a-89ea-4046-bee4-06e548243c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:19.437596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adadf3a-89ea-4046-bee4-06e548243c4f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:19.454295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 824/1638] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:19.466749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adadf3a-89ea-4046-bee4-06e548243c4f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:19.480656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7e1918774c4bbf9cbc83db40b0c85d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:19.481615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adadf3a-89ea-4046-bee4-06e548243c4f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:19.489782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc41f08-2bc8-45b3-ac4e-e867a7b4fa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:19.490008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7e1918774c4bbf9cbc83db40b0c85d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:19.490803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41f08-2bc8-45b3-ac4e-e867a7b4fa08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.491097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41f08-2bc8-45b3-ac4e-e867a7b4fa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.491375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41f08-2bc8-45b3-ac4e-e867a7b4fa08 admin admin] PATH_INFO: `/v3/projects/3e852f1933634f88affb5361a6b90cb1/users/1853c1f25be340158a0c84052e1f0c66/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.503518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7e1918774c4bbf9cbc83db40b0c85d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:19.513438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc41f08-2bc8-45b3-ac4e-e867a7b4fa08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e852f1933634f88affb5361a6b90cb1, user_id=1853c1f25be340158a0c84052e1f0c66, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:19.517571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc41f08-2bc8-45b3-ac4e-e867a7b4fa08 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:19.543439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7e1918774c4bbf9cbc83db40b0c85d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:19.550516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 825/1639] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:19 2022] PUT /identity/v3/projects/3e852f1933634f88affb5361a6b90cb1/users/1853c1f25be340158a0c84052e1f0c66/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:19.553997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6cd3ccc5-3d2b-4872-9fb8-ba647904344b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7e1918774c4bbf9cbc83db40b0c85d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:19.562562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e06db-e2f9-450a-a8ed-0a61b9551982 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.562787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e06db-e2f9-450a-a8ed-0a61b9551982 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.563265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e06db-e2f9-450a-a8ed-0a61b9551982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.567145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 815/1640] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:19 2022] DELETE /identity/v3/users/bc7e1918774c4bbf9cbc83db40b0c85d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:19.570343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62741b3-a3c0-4a3b-a788-d92d429bcf46 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.570757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62741b3-a3c0-4a3b-a788-d92d429bcf46 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.571043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62741b3-a3c0-4a3b-a788-d92d429bcf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.599969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-c62741b3-a3c0-4a3b-a788-d92d429bcf46 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:19.601442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c62741b3-a3c0-4a3b-a788-d92d429bcf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:19.627586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c60e06db-e2f9-450a-a8ed-0a61b9551982 None None] MFA Rules not processed for user `1853c1f25be340158a0c84052e1f0c66`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:19.632135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c60e06db-e2f9-450a-a8ed-0a61b9551982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:19.682271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:19.712575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 816/1641] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:19.726233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62741b3-a3c0-4a3b-a788-d92d429bcf46 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:19.728042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c62741b3-a3c0-4a3b-a788-d92d429bcf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:19.832458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 826/1642] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:19.837137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e06db-e2f9-450a-a8ed-0a61b9551982 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:19.841316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c60e06db-e2f9-450a-a8ed-0a61b9551982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:19.851255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62741b3-a3c0-4a3b-a788-d92d429bcf46 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:19.867460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f77fe-4b03-4487-9f52-2e08bb21a0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:19.867799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f77fe-4b03-4487-9f52-2e08bb21a0fc service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.868074 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f77fe-4b03-4487-9f52-2e08bb21a0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.869406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f77fe-4b03-4487-9f52-2e08bb21a0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.891639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f77fe-4b03-4487-9f52-2e08bb21a0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:19.896662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e06db-e2f9-450a-a8ed-0a61b9551982 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:19.903804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f77fe-4b03-4487-9f52-2e08bb21a0fc service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:19.912366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1152d-bd56-4f97-9975-bacc3169d149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:19.914053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1152d-bd56-4f97-9975-bacc3169d149 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:19.914419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1152d-bd56-4f97-9975-bacc3169d149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:19.914912 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1152d-bd56-4f97-9975-bacc3169d149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:19.915672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1152d-bd56-4f97-9975-bacc3169d149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:19.918108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1152d-bd56-4f97-9975-bacc3169d149 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:19.937112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 817/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:20.006908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f77fe-4b03-4487-9f52-2e08bb21a0fc service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:20.008336 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-323f77fe-4b03-4487-9f52-2e08bb21a0fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:20.009712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 827/1644] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:19 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:41:20.014441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1152d-bd56-4f97-9975-bacc3169d149 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:20.031978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f77fe-4b03-4487-9f52-2e08bb21a0fc service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:20.152990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:20.154673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:20.155217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:20.155694 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] PATH_INFO: `/v3/users/2ef4a1ca9ddc43f0b0a8271ed202178d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:20.157656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef4a1ca9ddc43f0b0a8271ed202178d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:20.160220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:20.171265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1152d-bd56-4f97-9975-bacc3169d149 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:20.192184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7180d-e7e2-4bca-b111-d414fab4b3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:20.193758 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7180d-e7e2-4bca-b111-d414fab4b3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:20.195131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7180d-e7e2-4bca-b111-d414fab4b3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:20.204628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7180d-e7e2-4bca-b111-d414fab4b3c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:20.205672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7180d-e7e2-4bca-b111-d414fab4b3c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:20.207748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7180d-e7e2-4bca-b111-d414fab4b3c0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:20.252956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 828/1645] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:20 2022] POST /identity/v3/users => generated 299 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:20.257802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7180d-e7e2-4bca-b111-d414fab4b3c0 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:20.290348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef4a1ca9ddc43f0b0a8271ed202178d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:20.300631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7180d-e7e2-4bca-b111-d414fab4b3c0 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:20.304607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef4a1ca9ddc43f0b0a8271ed202178d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:20.305966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef4a1ca9ddc43f0b0a8271ed202178d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:20.313634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e0943-3880-4bdb-a962-ceca2b748f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:20.314636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0943-3880-4bdb-a962-ceca2b748f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:20.315093 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0943-3880-4bdb-a962-ceca2b748f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:20.315457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0943-3880-4bdb-a962-ceca2b748f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:20.316314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e0943-3880-4bdb-a962-ceca2b748f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:20.326708 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e0943-3880-4bdb-a962-ceca2b748f1c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:20.332084 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef4a1ca9ddc43f0b0a8271ed202178d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:20.345723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef4a1ca9ddc43f0b0a8271ed202178d'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:20.352829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 829/1646] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:20.356324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 818/1647] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:20 2022] DELETE /identity/v3/users/2ef4a1ca9ddc43f0b0a8271ed202178d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:20.359469 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:20.361145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f89ec-e863-4470-864c-5e0b3c9d51ed None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:20.361436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f89ec-e863-4470-864c-5e0b3c9d51ed None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:20.361713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f89ec-e863-4470-864c-5e0b3c9d51ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:20.362075 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:20.407463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:20.439206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:20.445851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-501f89ec-e863-4470-864c-5e0b3c9d51ed None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:20.448365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-501f89ec-e863-4470-864c-5e0b3c9d51ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:20.550891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc288da-27a3-48bf-b2c6-caa51ecbe145 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:20.563225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e2dc0-fe4e-440a-91d8-5a673eba79dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:20.564000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e2dc0-fe4e-440a-91d8-5a673eba79dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:20.564298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e2dc0-fe4e-440a-91d8-5a673eba79dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:20.564828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e2dc0-fe4e-440a-91d8-5a673eba79dd admin admin] PATH_INFO: `/v3/projects/5f06ca74665045ff99b8cbf7c16bb704/users/78ae2a90883041a0a367f028ceb169f5/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:20.568880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 830/1648] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:20.569925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f89ec-e863-4470-864c-5e0b3c9d51ed None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:20.577077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-501f89ec-e863-4470-864c-5e0b3c9d51ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:20.587073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e2dc0-fe4e-440a-91d8-5a673eba79dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f06ca74665045ff99b8cbf7c16bb704, user_id=78ae2a90883041a0a367f028ceb169f5, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:20.591216 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e2dc0-fe4e-440a-91d8-5a673eba79dd admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:20.595089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f89ec-e863-4470-864c-5e0b3c9d51ed None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:20.601567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:20.602226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:20.602506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:20.602829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] PATH_INFO: `/v3/projects/01df70b705804ceca3a2e8cde5dd9372` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:20.604808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01df70b705804ceca3a2e8cde5dd9372)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:20.607304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:20.620440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 819/1649] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:20 2022] PUT /identity/v3/projects/5f06ca74665045ff99b8cbf7c16bb704/users/78ae2a90883041a0a367f028ceb169f5/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:20.620440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e2dc0-fe4e-440a-91d8-5a673eba79dd admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:20.623333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-012e2dc0-fe4e-440a-91d8-5a673eba79dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:20.677675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01df70b705804ceca3a2e8cde5dd9372', 'request_id': 'req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:20.687599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01df70b705804ceca3a2e8cde5dd9372', 'request_id': 'req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:20.698998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01df70b705804ceca3a2e8cde5dd9372', 'request_id': 'req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:20.706803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e2dc0-fe4e-440a-91d8-5a673eba79dd admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:20.716289 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] The token cache is being invalidate because project 01df70b705804ceca3a2e8cde5dd9372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:20.716685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:20.717607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:20.719947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 831/1650] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:20 2022] DELETE /identity/v3/projects/01df70b705804ceca3a2e8cde5dd9372 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:20.721269 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:20.725046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:20.787751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411cb90-73f5-4b78-acbc-36c21eceeaea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:20.788373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411cb90-73f5-4b78-acbc-36c21eceeaea service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:20.788891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411cb90-73f5-4b78-acbc-36c21eceeaea service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:20.789416 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411cb90-73f5-4b78-acbc-36c21eceeaea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:20.792185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7411cb90-73f5-4b78-acbc-36c21eceeaea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:20.815849 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e40e99-935e-4999-8e9e-fefbc8e45ec3 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:20.823513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170897e4-9200-48c0-bbac-e4019def7979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:20.824279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170897e4-9200-48c0-bbac-e4019def7979 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:20.824567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170897e4-9200-48c0-bbac-e4019def7979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:20.825005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170897e4-9200-48c0-bbac-e4019def7979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:20.825825 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170897e4-9200-48c0-bbac-e4019def7979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:20.828172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170897e4-9200-48c0-bbac-e4019def7979 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:20.842957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 832/1651] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:20.843826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170897e4-9200-48c0-bbac-e4019def7979 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:20.854673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-170897e4-9200-48c0-bbac-e4019def7979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:20.883715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170897e4-9200-48c0-bbac-e4019def7979 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:20.900257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411cb90-73f5-4b78-acbc-36c21eceeaea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:20.902555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411cb90-73f5-4b78-acbc-36c21eceeaea service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:20.912120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:20.913006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:20.913295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:20.913591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] PATH_INFO: `/v3/users/0be4b914da094e2d9f0f1b12f3c1db97` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:20.914009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 820/1652] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:20.914844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411cb90-73f5-4b78-acbc-36c21eceeaea service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:20.915117 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be4b914da094e2d9f0f1b12f3c1db97)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:20.917937 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:20.926649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411cb90-73f5-4b78-acbc-36c21eceeaea service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:20.932663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7820dc-a936-4bc0-be04-8970b6cfe2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:20.933821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7820dc-a936-4bc0-be04-8970b6cfe2de admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:20.934083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7820dc-a936-4bc0-be04-8970b6cfe2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:20.934318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7820dc-a936-4bc0-be04-8970b6cfe2de admin admin] PATH_INFO: `/v3/projects/5f06ca74665045ff99b8cbf7c16bb704/users/78ae2a90883041a0a367f028ceb169f5/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:20.940837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7820dc-a936-4bc0-be04-8970b6cfe2de admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f06ca74665045ff99b8cbf7c16bb704, user_id=78ae2a90883041a0a367f028ceb169f5, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:20.944676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7820dc-a936-4bc0-be04-8970b6cfe2de admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:20.946811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:20.953880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:20.981660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 821/1653] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:20 2022] PUT /identity/v3/projects/5f06ca74665045ff99b8cbf7c16bb704/users/78ae2a90883041a0a367f028ceb169f5/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:21.000476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be4b914da094e2d9f0f1b12f3c1db97'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.002569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480ba3-2da4-40c1-9dcf-dabf76213e02 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:21.003207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480ba3-2da4-40c1-9dcf-dabf76213e02 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:21.004992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480ba3-2da4-40c1-9dcf-dabf76213e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:21.019426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be4b914da094e2d9f0f1b12f3c1db97'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.020519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be4b914da094e2d9f0f1b12f3c1db97'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.031028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be4b914da094e2d9f0f1b12f3c1db97'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.032356 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be4b914da094e2d9f0f1b12f3c1db97'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.048514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-5b480ba3-2da4-40c1-9dcf-dabf76213e02 None None] MFA Rules not processed for user `78ae2a90883041a0a367f028ceb169f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:21.049631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 833/1654] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:20 2022] DELETE /identity/v3/users/0be4b914da094e2d9f0f1b12f3c1db97 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:21.057059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5b480ba3-2da4-40c1-9dcf-dabf76213e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:21.155316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 822/1655] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:21.170312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:21.171960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:21.254370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d485b27-c04b-4d11-8b4c-769da9c9c4f1 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:21.260607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83edc-1f86-4c61-a962-49bbec12afa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:21.261371 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83edc-1f86-4c61-a962-49bbec12afa2 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:21.261637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83edc-1f86-4c61-a962-49bbec12afa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:21.261871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83edc-1f86-4c61-a962-49bbec12afa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:21.281654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa83edc-1f86-4c61-a962-49bbec12afa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:21.283815 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa83edc-1f86-4c61-a962-49bbec12afa2 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:21.297700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 834/1656] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:21.341561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b480ba3-2da4-40c1-9dcf-dabf76213e02 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:21.343001 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5b480ba3-2da4-40c1-9dcf-dabf76213e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:21.435084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83edc-1f86-4c61-a962-49bbec12afa2 service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:21.436464 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0fa83edc-1f86-4c61-a962-49bbec12afa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:21.505708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b480ba3-2da4-40c1-9dcf-dabf76213e02 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:21.513328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:21.514027 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:21.514445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:21.516030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] PATH_INFO: `/v3/projects/48cfa127668f46f0a33edf6bb9266e1d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:21.516539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48cfa127668f46f0a33edf6bb9266e1d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:21.519032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:21.546310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83edc-1f86-4c61-a962-49bbec12afa2 service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:21.552178 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:21.552895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:21.553181 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:21.553444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] PATH_INFO: `/v3/projects/60b0a7c2f491416a9694f7637d8211e2` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:21.557915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b0a7c2f491416a9694f7637d8211e2)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:21.560169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:21.606895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48cfa127668f46f0a33edf6bb9266e1d', 'request_id': 'req-7ca8bf6b-b90a-43db-b664-d2f21c993987'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.610260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:21.613365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b0a7c2f491416a9694f7637d8211e2', 'request_id': 'req-3f6e2566-2a1f-4667-9049-05c8e2316ee1'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.615831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48cfa127668f46f0a33edf6bb9266e1d', 'request_id': 'req-7ca8bf6b-b90a-43db-b664-d2f21c993987'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.623951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b0a7c2f491416a9694f7637d8211e2', 'request_id': 'req-3f6e2566-2a1f-4667-9049-05c8e2316ee1'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.625725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48cfa127668f46f0a33edf6bb9266e1d', 'request_id': 'req-7ca8bf6b-b90a-43db-b664-d2f21c993987'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.632120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b0a7c2f491416a9694f7637d8211e2', 'request_id': 'req-3f6e2566-2a1f-4667-9049-05c8e2316ee1'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.640210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] The token cache is being invalidate because project 48cfa127668f46f0a33edf6bb9266e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:21.640793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.641654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.643363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 823/1657] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:21 2022] DELETE /identity/v3/projects/48cfa127668f46f0a33edf6bb9266e1d => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:21.643984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:21.645342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:21.647105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] The token cache is being invalidate because project 60b0a7c2f491416a9694f7637d8211e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:21.647561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.648584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.650097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 835/1658] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:21 2022] DELETE /identity/v3/projects/60b0a7c2f491416a9694f7637d8211e2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:21.654045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:21.656785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:21.664121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:21.723022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8bf6b-b90a-43db-b664-d2f21c993987 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:21.742483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e2566-2a1f-4667-9049-05c8e2316ee1 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:21.748116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:21.748800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:21.749112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:21.749389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] PATH_INFO: `/v3/users/45afbfc71abd4e7a8180d612eca34ae5` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:21.750521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45afbfc71abd4e7a8180d612eca34ae5)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:21.752523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:21.792095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:21.792982 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:21.793356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:21.794023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] PATH_INFO: `/v3/users/1853c1f25be340158a0c84052e1f0c66` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:21.795465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1853c1f25be340158a0c84052e1f0c66)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:21.797483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:21.810340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45afbfc71abd4e7a8180d612eca34ae5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.827887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45afbfc71abd4e7a8180d612eca34ae5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.829053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45afbfc71abd4e7a8180d612eca34ae5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.837165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45afbfc71abd4e7a8180d612eca34ae5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.837904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-24f2a001-f75e-4710-bb71-72280c326263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45afbfc71abd4e7a8180d612eca34ae5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.851858 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1853c1f25be340158a0c84052e1f0c66'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.854301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 836/1659] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:21 2022] DELETE /identity/v3/users/45afbfc71abd4e7a8180d612eca34ae5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:21.858866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930452f-92e0-4c71-b72f-62efbb208c17 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:21.859223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930452f-92e0-4c71-b72f-62efbb208c17 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:21.859507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930452f-92e0-4c71-b72f-62efbb208c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:21.872512 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1853c1f25be340158a0c84052e1f0c66'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.874864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1853c1f25be340158a0c84052e1f0c66'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.891820 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e930452f-92e0-4c71-b72f-62efbb208c17 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:21.893094 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e930452f-92e0-4c71-b72f-62efbb208c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:21.896738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1853c1f25be340158a0c84052e1f0c66'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.910605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-64eea6fe-e7ea-4847-9d76-d7ab881ec345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1853c1f25be340158a0c84052e1f0c66'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:21.921372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 824/1660] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:21 2022] DELETE /identity/v3/users/1853c1f25be340158a0c84052e1f0c66 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:21.928181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30852-3657-4d09-a15a-eb64c00bcc4e None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:21.929022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30852-3657-4d09-a15a-eb64c00bcc4e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:21.929304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30852-3657-4d09-a15a-eb64c00bcc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:21.973522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-00c30852-3657-4d09-a15a-eb64c00bcc4e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:21.973522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-00c30852-3657-4d09-a15a-eb64c00bcc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:21.979255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:22.003033 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 837/1661] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:22.012801 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb86e19-3075-4e3d-a5f2-6c2097fe49a4 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.013140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb86e19-3075-4e3d-a5f2-6c2097fe49a4 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.013425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb86e19-3075-4e3d-a5f2-6c2097fe49a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.013618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:22.048097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-3bb86e19-3075-4e3d-a5f2-6c2097fe49a4 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:22.049496 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3bb86e19-3075-4e3d-a5f2-6c2097fe49a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.054927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 825/1662] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:22.066922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 838/1663] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:22.072177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c30852-3657-4d09-a15a-eb64c00bcc4e None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.073729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb86e19-3075-4e3d-a5f2-6c2097fe49a4 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.073898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-00c30852-3657-4d09-a15a-eb64c00bcc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.084217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb86e19-3075-4e3d-a5f2-6c2097fe49a4 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.092038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289ff7b-47f0-42a1-9721-3abf2ad612d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.092695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289ff7b-47f0-42a1-9721-3abf2ad612d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.093205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289ff7b-47f0-42a1-9721-3abf2ad612d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.093486 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289ff7b-47f0-42a1-9721-3abf2ad612d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.094164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289ff7b-47f0-42a1-9721-3abf2ad612d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:22.094581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289ff7b-47f0-42a1-9721-3abf2ad612d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.096889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289ff7b-47f0-42a1-9721-3abf2ad612d5 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.110619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 839/1664] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:22.115575 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289ff7b-47f0-42a1-9721-3abf2ad612d5 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.131443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289ff7b-47f0-42a1-9721-3abf2ad612d5 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.138309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9febd-40ed-443b-92b6-13be001334d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.138935 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9febd-40ed-443b-92b6-13be001334d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.139212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9febd-40ed-443b-92b6-13be001334d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.139500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9febd-40ed-443b-92b6-13be001334d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.140104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9febd-40ed-443b-92b6-13be001334d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.142193 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9febd-40ed-443b-92b6-13be001334d6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.157315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c30852-3657-4d09-a15a-eb64c00bcc4e None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.162803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47df71-dfe2-4dc9-81c4-90f44e898809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.163544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47df71-dfe2-4dc9-81c4-90f44e898809 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.163739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47df71-dfe2-4dc9-81c4-90f44e898809 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.164031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47df71-dfe2-4dc9-81c4-90f44e898809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.175442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47df71-dfe2-4dc9-81c4-90f44e898809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.177705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47df71-dfe2-4dc9-81c4-90f44e898809 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.184585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 840/1665] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:22 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:22.185864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9febd-40ed-443b-92b6-13be001334d6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.192829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 826/1666] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:22.279446 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9febd-40ed-443b-92b6-13be001334d6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.286659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5b9d5-2560-4418-bd80-a0b65f12a390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.287397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5b9d5-2560-4418-bd80-a0b65f12a390 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.287750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5b9d5-2560-4418-bd80-a0b65f12a390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.288096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5b9d5-2560-4418-bd80-a0b65f12a390 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.288914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5b9d5-2560-4418-bd80-a0b65f12a390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.292330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5b9d5-2560-4418-bd80-a0b65f12a390 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.307315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47df71-dfe2-4dc9-81c4-90f44e898809 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.310556 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fb47df71-dfe2-4dc9-81c4-90f44e898809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.333029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47df71-dfe2-4dc9-81c4-90f44e898809 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.340631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.341849 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.342190 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.342589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] PATH_INFO: `/v3/projects/04f7d5a87b214ff9a73cd31199e5e0d1` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.345182 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 841/1667] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:22 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:22.348699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5b9d5-2560-4418-bd80-a0b65f12a390 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.358089 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f7d5a87b214ff9a73cd31199e5e0d1)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.365620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.366172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5b9d5-2560-4418-bd80-a0b65f12a390 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.376417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07b1a1-5851-4a9a-ba5a-a601adb1c407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.376609 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07b1a1-5851-4a9a-ba5a-a601adb1c407 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.376931 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07b1a1-5851-4a9a-ba5a-a601adb1c407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.377142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07b1a1-5851-4a9a-ba5a-a601adb1c407 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.377868 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07b1a1-5851-4a9a-ba5a-a601adb1c407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.379848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07b1a1-5851-4a9a-ba5a-a601adb1c407 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.391941 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 842/1668] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:22.396260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07b1a1-5851-4a9a-ba5a-a601adb1c407 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.410965 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07b1a1-5851-4a9a-ba5a-a601adb1c407 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.431650 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db73098-f617-42b3-926f-4368fd96015f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.436281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db73098-f617-42b3-926f-4368fd96015f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.436573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db73098-f617-42b3-926f-4368fd96015f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.437301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db73098-f617-42b3-926f-4368fd96015f admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.438073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db73098-f617-42b3-926f-4368fd96015f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.440161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db73098-f617-42b3-926f-4368fd96015f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.454569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 843/1669] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:41:22.458716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db73098-f617-42b3-926f-4368fd96015f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.467535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f7d5a87b214ff9a73cd31199e5e0d1', 'request_id': 'req-edb5caa8-8577-4ec0-8f97-e3fba3857538'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.475405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f7d5a87b214ff9a73cd31199e5e0d1', 'request_id': 'req-edb5caa8-8577-4ec0-8f97-e3fba3857538'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.477644 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db73098-f617-42b3-926f-4368fd96015f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.487035 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f7d5a87b214ff9a73cd31199e5e0d1', 'request_id': 'req-edb5caa8-8577-4ec0-8f97-e3fba3857538'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.502514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] The token cache is being invalidate because project 04f7d5a87b214ff9a73cd31199e5e0d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:22.502864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.503485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-edb5caa8-8577-4ec0-8f97-e3fba3857538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.505178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 827/1670] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:22 2022] DELETE /identity/v3/projects/04f7d5a87b214ff9a73cd31199e5e0d1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:22.509839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7d5c9-4961-4f2b-b6e7-7e0188628d13 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.510694 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7d5c9-4961-4f2b-b6e7-7e0188628d13 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.510965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7d5c9-4961-4f2b-b6e7-7e0188628d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.535679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-5fc7d5c9-4961-4f2b-b6e7-7e0188628d13 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:22.537481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5fc7d5c9-4961-4f2b-b6e7-7e0188628d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.552266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23610425-c9eb-44d6-8d6e-a2c0a583a28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.552977 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23610425-c9eb-44d6-8d6e-a2c0a583a28b admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.553259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23610425-c9eb-44d6-8d6e-a2c0a583a28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.553532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23610425-c9eb-44d6-8d6e-a2c0a583a28b admin admin] PATH_INFO: `/v3/projects/e323a2101ba84b7697bb0546466021a6/users/af5bfb7a152f432198061c3538a71276/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.569691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23610425-c9eb-44d6-8d6e-a2c0a583a28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e323a2101ba84b7697bb0546466021a6, user_id=af5bfb7a152f432198061c3538a71276, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.572412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23610425-c9eb-44d6-8d6e-a2c0a583a28b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.584060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:22.599289 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 844/1671] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:22 2022] PUT /identity/v3/projects/e323a2101ba84b7697bb0546466021a6/users/af5bfb7a152f432198061c3538a71276/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:22.609596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ce116-ebf7-4460-af7a-00f4f9cc5704 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.610007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ce116-ebf7-4460-af7a-00f4f9cc5704 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.610291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ce116-ebf7-4460-af7a-00f4f9cc5704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.617462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 828/1672] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:22.627069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9296c-8e48-41fd-9881-61fe5e69c699 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.627404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9296c-8e48-41fd-9881-61fe5e69c699 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.627696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9296c-8e48-41fd-9881-61fe5e69c699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.637646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:22.657478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-ec7ce116-ebf7-4460-af7a-00f4f9cc5704 None None] MFA Rules not processed for user `af5bfb7a152f432198061c3538a71276`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:22.659404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ec7ce116-ebf7-4460-af7a-00f4f9cc5704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.672549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-76d9296c-8e48-41fd-9881-61fe5e69c699 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:22.674179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-76d9296c-8e48-41fd-9881-61fe5e69c699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.743647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 845/1673] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:22.745376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ce116-ebf7-4460-af7a-00f4f9cc5704 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.746676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ec7ce116-ebf7-4460-af7a-00f4f9cc5704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.755062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 829/1674] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:22.756815 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9296c-8e48-41fd-9881-61fe5e69c699 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.758816 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-76d9296c-8e48-41fd-9881-61fe5e69c699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.763608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ce116-ebf7-4460-af7a-00f4f9cc5704 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.768444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.769112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.769391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.769664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] PATH_INFO: `/v3/projects/3e852f1933634f88affb5361a6b90cb1` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.771029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e852f1933634f88affb5361a6b90cb1)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.772715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9296c-8e48-41fd-9881-61fe5e69c699 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.773105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.778483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdb78f-aae0-468a-8ab8-0bd16cb6730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.779200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdb78f-aae0-468a-8ab8-0bd16cb6730f admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.779654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdb78f-aae0-468a-8ab8-0bd16cb6730f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.779817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdb78f-aae0-468a-8ab8-0bd16cb6730f admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.780465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdb78f-aae0-468a-8ab8-0bd16cb6730f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.782759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdb78f-aae0-468a-8ab8-0bd16cb6730f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.815984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 830/1675] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:22 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:22.817101 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdb78f-aae0-468a-8ab8-0bd16cb6730f admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.831253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e852f1933634f88affb5361a6b90cb1', 'request_id': 'req-a85fe813-50c9-465d-aa59-eaa935be2cd0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.837021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e852f1933634f88affb5361a6b90cb1', 'request_id': 'req-a85fe813-50c9-465d-aa59-eaa935be2cd0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.845306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e852f1933634f88affb5361a6b90cb1', 'request_id': 'req-a85fe813-50c9-465d-aa59-eaa935be2cd0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.859476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] The token cache is being invalidate because project 3e852f1933634f88affb5361a6b90cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:22.860055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.860466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.862287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 846/1676] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:22 2022] DELETE /identity/v3/projects/3e852f1933634f88affb5361a6b90cb1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:22.862864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.864143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.886799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdb78f-aae0-468a-8ab8-0bd16cb6730f admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.891488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a963e-2bdd-48e9-97aa-d38ab48b3947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.892228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a963e-2bdd-48e9-97aa-d38ab48b3947 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.892506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a963e-2bdd-48e9-97aa-d38ab48b3947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.892780 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a963e-2bdd-48e9-97aa-d38ab48b3947 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.893461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a963e-2bdd-48e9-97aa-d38ab48b3947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:22.893761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a963e-2bdd-48e9-97aa-d38ab48b3947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.895348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a963e-2bdd-48e9-97aa-d38ab48b3947 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.907105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 831/1677] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:22.907938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a963e-2bdd-48e9-97aa-d38ab48b3947 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.911795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fd0a963e-2bdd-48e9-97aa-d38ab48b3947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.925306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a963e-2bdd-48e9-97aa-d38ab48b3947 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.929975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.930938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.930938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.931171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] PATH_INFO: `/v3/users/78ae2a90883041a0a367f028ceb169f5` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.932891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ae2a90883041a0a367f028ceb169f5)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.934823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.936061 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe813-50c9-465d-aa59-eaa935be2cd0 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.940830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682b9ce-130b-4420-a0b0-d5e5f9461c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.941466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b9ce-130b-4420-a0b0-d5e5f9461c38 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.941743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b9ce-130b-4420-a0b0-d5e5f9461c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.942015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b9ce-130b-4420-a0b0-d5e5f9461c38 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.942703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682b9ce-130b-4420-a0b0-d5e5f9461c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.944622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682b9ce-130b-4420-a0b0-d5e5f9461c38 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.968545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 847/1678] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:22 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:22.969431 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682b9ce-130b-4420-a0b0-d5e5f9461c38 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:22.971017 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6682b9ce-130b-4420-a0b0-d5e5f9461c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:22.978597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:22.986450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682b9ce-130b-4420-a0b0-d5e5f9461c38 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:22.987633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ae2a90883041a0a367f028ceb169f5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:22.992754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd8e59-e019-49bc-b2f0-c8249d5b1eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:22.993439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd8e59-e019-49bc-b2f0-c8249d5b1eac admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:22.993729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd8e59-e019-49bc-b2f0-c8249d5b1eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:22.994006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd8e59-e019-49bc-b2f0-c8249d5b1eac admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:22.995826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd8e59-e019-49bc-b2f0-c8249d5b1eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:22.998298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd8e59-e019-49bc-b2f0-c8249d5b1eac admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:22.998898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ae2a90883041a0a367f028ceb169f5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:23.000418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ae2a90883041a0a367f028ceb169f5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:23.012433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:23.015004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ae2a90883041a0a367f028ceb169f5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:23.037519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ae2a90883041a0a367f028ceb169f5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:23.044660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 848/1679] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:22 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:23.044660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd8e59-e019-49bc-b2f0-c8249d5b1eac admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.044660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-58fd8e59-e019-49bc-b2f0-c8249d5b1eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:23.051177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 832/1680] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:22 2022] DELETE /identity/v3/users/78ae2a90883041a0a367f028ceb169f5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:23.051909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.055569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:23.117837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd8e59-e019-49bc-b2f0-c8249d5b1eac admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.139553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b047b-8c30-4fa1-a9d8-bdc6c2884df0 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.145671 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477be02f-5918-4405-8802-a21f38d7edb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.146297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477be02f-5918-4405-8802-a21f38d7edb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.146578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477be02f-5918-4405-8802-a21f38d7edb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.147258 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477be02f-5918-4405-8802-a21f38d7edb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.147486 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477be02f-5918-4405-8802-a21f38d7edb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.150120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477be02f-5918-4405-8802-a21f38d7edb4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.184536 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 833/1681] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:23 2022] POST /identity/v3/users => generated 284 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:23.188042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc5cc4-f256-440e-8542-6ff509225ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.188230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477be02f-5918-4405-8802-a21f38d7edb4 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.188887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5cc4-f256-440e-8542-6ff509225ddb admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.189065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5cc4-f256-440e-8542-6ff509225ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.189333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5cc4-f256-440e-8542-6ff509225ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.189979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc5cc4-f256-440e-8542-6ff509225ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.191566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc5cc4-f256-440e-8542-6ff509225ddb admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.200518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477be02f-5918-4405-8802-a21f38d7edb4 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.207176 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 849/1682] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:23.208198 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326644e-11a2-476f-aee1-866cdf07ab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.208328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc5cc4-f256-440e-8542-6ff509225ddb admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.209077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326644e-11a2-476f-aee1-866cdf07ab89 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.209491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326644e-11a2-476f-aee1-866cdf07ab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.209827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326644e-11a2-476f-aee1-866cdf07ab89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.210492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326644e-11a2-476f-aee1-866cdf07ab89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.212150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326644e-11a2-476f-aee1-866cdf07ab89 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.219086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc5cc4-f256-440e-8542-6ff509225ddb admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.225157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116e984-ba48-4317-a289-3d91c97bd28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.225803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116e984-ba48-4317-a289-3d91c97bd28c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.226077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116e984-ba48-4317-a289-3d91c97bd28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.226349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116e984-ba48-4317-a289-3d91c97bd28c admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.226986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116e984-ba48-4317-a289-3d91c97bd28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.228572 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116e984-ba48-4317-a289-3d91c97bd28c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.230177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 834/1683] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:41:23.233962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326644e-11a2-476f-aee1-866cdf07ab89 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.242700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 850/1684] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:23.246994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326644e-11a2-476f-aee1-866cdf07ab89 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.247368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116e984-ba48-4317-a289-3d91c97bd28c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.252876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeae2f8-2ac8-4c53-9f6b-93f319a33d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.253490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeae2f8-2ac8-4c53-9f6b-93f319a33d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.253774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeae2f8-2ac8-4c53-9f6b-93f319a33d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.254566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeae2f8-2ac8-4c53-9f6b-93f319a33d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.255067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeae2f8-2ac8-4c53-9f6b-93f319a33d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.256927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeae2f8-2ac8-4c53-9f6b-93f319a33d44 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.257559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116e984-ba48-4317-a289-3d91c97bd28c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.263023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd61ce-fb6d-4dec-9ecd-15017f9be8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.263691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd61ce-fb6d-4dec-9ecd-15017f9be8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.263968 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd61ce-fb6d-4dec-9ecd-15017f9be8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.264238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd61ce-fb6d-4dec-9ecd-15017f9be8fe admin admin] PATH_INFO: `/v3/projects/daf44851f7424d85aa3bcd66ab122aa9/users/29e869c440164a82946db0eafb8c62e4/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.274494 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 835/1685] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:41:23.278466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeae2f8-2ac8-4c53-9f6b-93f319a33d44 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.284969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd61ce-fb6d-4dec-9ecd-15017f9be8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf44851f7424d85aa3bcd66ab122aa9, user_id=29e869c440164a82946db0eafb8c62e4, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.288441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd61ce-fb6d-4dec-9ecd-15017f9be8fe admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.291886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeae2f8-2ac8-4c53-9f6b-93f319a33d44 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.299247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee61d1-f73c-499d-85ba-fc7119683e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.300234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee61d1-f73c-499d-85ba-fc7119683e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.300748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee61d1-f73c-499d-85ba-fc7119683e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.301251 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee61d1-f73c-499d-85ba-fc7119683e6a admin admin] PATH_INFO: `/v3/projects/7d9a90f4e79d4b8eaf2f1da23ca9c5db/users/34557b30965747c496c294e2474039c2/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.303502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 851/1686] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:23 2022] PUT /identity/v3/projects/daf44851f7424d85aa3bcd66ab122aa9/users/29e869c440164a82946db0eafb8c62e4/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:23.315459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0fb07-4069-4270-8c46-109af52c80f5 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.315738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0fb07-4069-4270-8c46-109af52c80f5 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.316028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0fb07-4069-4270-8c46-109af52c80f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.320995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee61d1-f73c-499d-85ba-fc7119683e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9a90f4e79d4b8eaf2f1da23ca9c5db, user_id=34557b30965747c496c294e2474039c2, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.324527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee61d1-f73c-499d-85ba-fc7119683e6a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.343951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 836/1687] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:23 2022] PUT /identity/v3/projects/7d9a90f4e79d4b8eaf2f1da23ca9c5db/users/34557b30965747c496c294e2474039c2/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:23.353597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-f9c0fb07-4069-4270-8c46-109af52c80f5 None None] MFA Rules not processed for user `29e869c440164a82946db0eafb8c62e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:23.353597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f9c0fb07-4069-4270-8c46-109af52c80f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:23.356883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4dd7e-7391-4fcc-b02b-32c2dda896d6 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.357354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4dd7e-7391-4fcc-b02b-32c2dda896d6 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.357654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4dd7e-7391-4fcc-b02b-32c2dda896d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.393061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-78c4dd7e-7391-4fcc-b02b-32c2dda896d6 None None] MFA Rules not processed for user `34557b30965747c496c294e2474039c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:23.394487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-78c4dd7e-7391-4fcc-b02b-32c2dda896d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:23.405685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:23.436444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 852/1688] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:23.437325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0fb07-4069-4270-8c46-109af52c80f5 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.439138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f9c0fb07-4069-4270-8c46-109af52c80f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:23.470487 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 837/1689] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:23.471348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4dd7e-7391-4fcc-b02b-32c2dda896d6 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.492702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:23.512972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0fb07-4069-4270-8c46-109af52c80f5 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.517758 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.518406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.518690 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.518962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] PATH_INFO: `/v3/projects/5f06ca74665045ff99b8cbf7c16bb704` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.520104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f06ca74665045ff99b8cbf7c16bb704)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.522081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.547459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4dd7e-7391-4fcc-b02b-32c2dda896d6 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.552766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7cae4-f2d3-42f5-b72e-776da3c9705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.554021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7cae4-f2d3-42f5-b72e-776da3c9705e admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.554627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7cae4-f2d3-42f5-b72e-776da3c9705e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.555152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7cae4-f2d3-42f5-b72e-776da3c9705e admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.557789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7cae4-f2d3-42f5-b72e-776da3c9705e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.559880 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7cae4-f2d3-42f5-b72e-776da3c9705e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.574530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f06ca74665045ff99b8cbf7c16bb704', 'request_id': 'req-df1a37da-476e-44da-8800-b6ea02bad63d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:23.581949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f06ca74665045ff99b8cbf7c16bb704', 'request_id': 'req-df1a37da-476e-44da-8800-b6ea02bad63d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:23.590244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f06ca74665045ff99b8cbf7c16bb704', 'request_id': 'req-df1a37da-476e-44da-8800-b6ea02bad63d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:23.593643 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 838/1690] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:23 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:23.597325 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7cae4-f2d3-42f5-b72e-776da3c9705e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.604108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] The token cache is being invalidate because project 5f06ca74665045ff99b8cbf7c16bb704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:23.604508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:23.605213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:23.606868 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 853/1691] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:23 2022] DELETE /identity/v3/projects/5f06ca74665045ff99b8cbf7c16bb704 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:23.607624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.608994 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:23.671461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7cae4-f2d3-42f5-b72e-776da3c9705e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.676643 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95966e-bc6e-4b30-89ed-dbb4419fb36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.677304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95966e-bc6e-4b30-89ed-dbb4419fb36b admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.677583 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95966e-bc6e-4b30-89ed-dbb4419fb36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.677866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95966e-bc6e-4b30-89ed-dbb4419fb36b admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.678504 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95966e-bc6e-4b30-89ed-dbb4419fb36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.680428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95966e-bc6e-4b30-89ed-dbb4419fb36b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.689948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a37da-476e-44da-8800-b6ea02bad63d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.695057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda6888-243b-4298-a5cb-1b0ed16a5c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.695735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda6888-243b-4298-a5cb-1b0ed16a5c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.696177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda6888-243b-4298-a5cb-1b0ed16a5c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.696517 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda6888-243b-4298-a5cb-1b0ed16a5c49 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.697265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda6888-243b-4298-a5cb-1b0ed16a5c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.700071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda6888-243b-4298-a5cb-1b0ed16a5c49 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.711975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 839/1692] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:23 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:23.717237 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831ee95-95a1-467a-a49d-6588ccd91890 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.717525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831ee95-95a1-467a-a49d-6588ccd91890 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.717818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831ee95-95a1-467a-a49d-6588ccd91890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.719588 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:23.726572 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 854/1693] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:23 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:23.727555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda6888-243b-4298-a5cb-1b0ed16a5c49 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.730588 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2dda6888-243b-4298-a5cb-1b0ed16a5c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:23.739687 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-0831ee95-95a1-467a-a49d-6588ccd91890 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:23.740944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0831ee95-95a1-467a-a49d-6588ccd91890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:23.744940 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda6888-243b-4298-a5cb-1b0ed16a5c49 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.812321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc164d3c-1b6e-4e4f-945b-a988fcfdc964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.812983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc164d3c-1b6e-4e4f-945b-a988fcfdc964 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.813267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc164d3c-1b6e-4e4f-945b-a988fcfdc964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.813540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc164d3c-1b6e-4e4f-945b-a988fcfdc964 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.814187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc164d3c-1b6e-4e4f-945b-a988fcfdc964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.821945 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc164d3c-1b6e-4e4f-945b-a988fcfdc964 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.822166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 840/1694] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:23.822166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831ee95-95a1-467a-a49d-6588ccd91890 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.832890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831ee95-95a1-467a-a49d-6588ccd91890 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.856817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 855/1695] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:23 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:23.858177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49a7b0-0bde-4a2b-be3a-1fffb9d2aaa8 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.858467 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49a7b0-0bde-4a2b-be3a-1fffb9d2aaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.858742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49a7b0-0bde-4a2b-be3a-1fffb9d2aaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.860155 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a240bf3-b31b-4c06-919c-c6e906d2b9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.860793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a240bf3-b31b-4c06-919c-c6e906d2b9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.861103 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a240bf3-b31b-4c06-919c-c6e906d2b9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.861376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a240bf3-b31b-4c06-919c-c6e906d2b9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.862028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a240bf3-b31b-4c06-919c-c6e906d2b9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.863644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a240bf3-b31b-4c06-919c-c6e906d2b9bf admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.877972 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 841/1696] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:23.879407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a240bf3-b31b-4c06-919c-c6e906d2b9bf admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.885217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-4d49a7b0-0bde-4a2b-be3a-1fffb9d2aaa8 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:23.886507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4d49a7b0-0bde-4a2b-be3a-1fffb9d2aaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:23.897847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 856/1697] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:23.897847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49a7b0-0bde-4a2b-be3a-1fffb9d2aaa8 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.915025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49a7b0-0bde-4a2b-be3a-1fffb9d2aaa8 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.918409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a240bf3-b31b-4c06-919c-c6e906d2b9bf admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.921584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb23ba6-76ad-44fc-94ea-e1b6dbb417d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.922462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb23ba6-76ad-44fc-94ea-e1b6dbb417d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.922748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb23ba6-76ad-44fc-94ea-e1b6dbb417d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.923028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb23ba6-76ad-44fc-94ea-e1b6dbb417d4 admin admin] PATH_INFO: `/v3/projects/37f33517247e412eb0e83586644e433f/users/226b9b89fec844219427b825eac1eed0/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.935902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faaeeac-4a19-4e14-98e2-b39e9d345ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:23.938369 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaeeac-4a19-4e14-98e2-b39e9d345ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:23.938887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaeeac-4a19-4e14-98e2-b39e9d345ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:23.939430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaeeac-4a19-4e14-98e2-b39e9d345ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:23.939964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faaeeac-4a19-4e14-98e2-b39e9d345ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.941649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faaeeac-4a19-4e14-98e2-b39e9d345ea3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.944472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb23ba6-76ad-44fc-94ea-e1b6dbb417d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f33517247e412eb0e83586644e433f, user_id=226b9b89fec844219427b825eac1eed0, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:23.947896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb23ba6-76ad-44fc-94ea-e1b6dbb417d4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:23.961959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 842/1698] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:23.963474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faaeeac-4a19-4e14-98e2-b39e9d345ea3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.967658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 857/1699] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:23 2022] PUT /identity/v3/projects/37f33517247e412eb0e83586644e433f/users/226b9b89fec844219427b825eac1eed0/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:23.971796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb23ba6-76ad-44fc-94ea-e1b6dbb417d4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:23.975170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faaeeac-4a19-4e14-98e2-b39e9d345ea3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:23.981235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:24.057321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc256-2218-4ac7-8459-b8d0ab7e570e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.058468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc256-2218-4ac7-8459-b8d0ab7e570e admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.059394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc256-2218-4ac7-8459-b8d0ab7e570e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.059677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfc256-2218-4ac7-8459-b8d0ab7e570e admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.060346 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfc256-2218-4ac7-8459-b8d0ab7e570e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:24.060658 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfc256-2218-4ac7-8459-b8d0ab7e570e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.062411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfc256-2218-4ac7-8459-b8d0ab7e570e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.069101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb23ba6-76ad-44fc-94ea-e1b6dbb417d4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.087912 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726748f4-6088-4dae-b89e-6244f55d647f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.088936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 843/1700] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:24.089873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc256-2218-4ac7-8459-b8d0ab7e570e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.090825 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726748f4-6088-4dae-b89e-6244f55d647f admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.091111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726748f4-6088-4dae-b89e-6244f55d647f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.091459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726748f4-6088-4dae-b89e-6244f55d647f admin admin] PATH_INFO: `/v3/projects/8fd355c23e8c4519a4d44cfbb6462f7e/users/e78383ff34fc48f8b1d3843fdde1533b/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.106830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfc256-2218-4ac7-8459-b8d0ab7e570e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.114363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e5126-f903-4de9-ae96-d69fd43de330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.115271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e5126-f903-4de9-ae96-d69fd43de330 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.115703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e5126-f903-4de9-ae96-d69fd43de330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.116176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e5126-f903-4de9-ae96-d69fd43de330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.116629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726748f4-6088-4dae-b89e-6244f55d647f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd355c23e8c4519a4d44cfbb6462f7e, user_id=e78383ff34fc48f8b1d3843fdde1533b, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.117251 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e5126-f903-4de9-ae96-d69fd43de330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.120023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e5126-f903-4de9-ae96-d69fd43de330 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.120765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726748f4-6088-4dae-b89e-6244f55d647f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.145180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 844/1701] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:24.145180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e5126-f903-4de9-ae96-d69fd43de330 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.148687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 858/1702] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:23 2022] PUT /identity/v3/projects/8fd355c23e8c4519a4d44cfbb6462f7e/users/e78383ff34fc48f8b1d3843fdde1533b/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:24.153506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726748f4-6088-4dae-b89e-6244f55d647f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.159378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e5126-f903-4de9-ae96-d69fd43de330 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.232344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:24.251633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8124f1-b511-4ef8-9688-38f07cbc0c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.251633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8124f1-b511-4ef8-9688-38f07cbc0c29 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.251633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8124f1-b511-4ef8-9688-38f07cbc0c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.251633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8124f1-b511-4ef8-9688-38f07cbc0c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.252050 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8124f1-b511-4ef8-9688-38f07cbc0c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.253824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8124f1-b511-4ef8-9688-38f07cbc0c29 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.280521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726748f4-6088-4dae-b89e-6244f55d647f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.285495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f638d6-47d4-40fc-8860-bbdd9930e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.286164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f638d6-47d4-40fc-8860-bbdd9930e9e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.286440 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f638d6-47d4-40fc-8860-bbdd9930e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.286709 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f638d6-47d4-40fc-8860-bbdd9930e9e8 admin admin] PATH_INFO: `/v3/projects/37f33517247e412eb0e83586644e433f/users/226b9b89fec844219427b825eac1eed0/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.288970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f638d6-47d4-40fc-8860-bbdd9930e9e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f33517247e412eb0e83586644e433f, user_id=226b9b89fec844219427b825eac1eed0, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.291910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 845/1703] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:24 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:24.293007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8124f1-b511-4ef8-9688-38f07cbc0c29 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.293194 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f638d6-47d4-40fc-8860-bbdd9930e9e8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.309192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 859/1704] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:24 2022] PUT /identity/v3/projects/37f33517247e412eb0e83586644e433f/users/226b9b89fec844219427b825eac1eed0/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:24.309192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f638d6-47d4-40fc-8860-bbdd9930e9e8 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.386522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8124f1-b511-4ef8-9688-38f07cbc0c29 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.391563 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c0416-634c-4dc2-82d7-106dc6a36c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.392179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c0416-634c-4dc2-82d7-106dc6a36c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.393231 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c0416-634c-4dc2-82d7-106dc6a36c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.393464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c0416-634c-4dc2-82d7-106dc6a36c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.394049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f638d6-47d4-40fc-8860-bbdd9930e9e8 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.394166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c0416-634c-4dc2-82d7-106dc6a36c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.395825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c0416-634c-4dc2-82d7-106dc6a36c99 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.399442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939241a3-f086-4d8e-8cc2-70dea246301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.401096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939241a3-f086-4d8e-8cc2-70dea246301d admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.401303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939241a3-f086-4d8e-8cc2-70dea246301d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.401688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939241a3-f086-4d8e-8cc2-70dea246301d admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.402424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939241a3-f086-4d8e-8cc2-70dea246301d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.404439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939241a3-f086-4d8e-8cc2-70dea246301d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.407121 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 846/1705] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:24.408252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03a900-15f4-4412-bfea-3200fea8e870 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.408528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03a900-15f4-4412-bfea-3200fea8e870 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.408794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03a900-15f4-4412-bfea-3200fea8e870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.436199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 860/1706] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:24 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:24.436199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939241a3-f086-4d8e-8cc2-70dea246301d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.437870 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:24.440000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-5b03a900-15f4-4412-bfea-3200fea8e870 None None] MFA Rules not processed for user `226b9b89fec844219427b825eac1eed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:24.442793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5b03a900-15f4-4412-bfea-3200fea8e870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:24.450600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939241a3-f086-4d8e-8cc2-70dea246301d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.456812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188cbb3-c41e-483b-9004-bddad9fd2b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.464227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188cbb3-c41e-483b-9004-bddad9fd2b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.464619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188cbb3-c41e-483b-9004-bddad9fd2b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.465075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188cbb3-c41e-483b-9004-bddad9fd2b8c admin admin] PATH_INFO: `/v3/projects/8fd355c23e8c4519a4d44cfbb6462f7e/users/e78383ff34fc48f8b1d3843fdde1533b/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.468764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188cbb3-c41e-483b-9004-bddad9fd2b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd355c23e8c4519a4d44cfbb6462f7e, user_id=e78383ff34fc48f8b1d3843fdde1533b, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.472863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188cbb3-c41e-483b-9004-bddad9fd2b8c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.486626 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 861/1707] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:24 2022] PUT /identity/v3/projects/8fd355c23e8c4519a4d44cfbb6462f7e/users/e78383ff34fc48f8b1d3843fdde1533b/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:24.487548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188cbb3-c41e-483b-9004-bddad9fd2b8c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.542247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 847/1708] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:24.543573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3697152-5f5d-4b28-a15d-27c820daa760 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.543853 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3697152-5f5d-4b28-a15d-27c820daa760 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.544126 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3697152-5f5d-4b28-a15d-27c820daa760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.561192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188cbb3-c41e-483b-9004-bddad9fd2b8c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.566692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbeb5a-2145-4028-8a78-cc126ab0a291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.568188 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbeb5a-2145-4028-8a78-cc126ab0a291 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.568469 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbeb5a-2145-4028-8a78-cc126ab0a291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.568757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbeb5a-2145-4028-8a78-cc126ab0a291 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.569431 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbeb5a-2145-4028-8a78-cc126ab0a291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.571038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbeb5a-2145-4028-8a78-cc126ab0a291 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.576095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-c3697152-5f5d-4b28-a15d-27c820daa760 None None] MFA Rules not processed for user `e78383ff34fc48f8b1d3843fdde1533b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:24.577630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c3697152-5f5d-4b28-a15d-27c820daa760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:24.583893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 862/1709] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:24.585039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbeb5a-2145-4028-8a78-cc126ab0a291 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.586870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-27cbeb5a-2145-4028-8a78-cc126ab0a291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:24.645061 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:24.652417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbeb5a-2145-4028-8a78-cc126ab0a291 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.656483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c280a-c96a-420e-ba9b-41896c1174c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.657147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c280a-c96a-420e-ba9b-41896c1174c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.657428 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c280a-c96a-420e-ba9b-41896c1174c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.657858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c280a-c96a-420e-ba9b-41896c1174c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.659867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5f5c280a-c96a-420e-ba9b-41896c1174c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:24.664894 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 848/1710] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:24.665761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3697152-5f5d-4b28-a15d-27c820daa760 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.683497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3697152-5f5d-4b28-a15d-27c820daa760 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.690139 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caa70c-0db4-41f5-b77e-07a598b4aaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.691147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caa70c-0db4-41f5-b77e-07a598b4aaa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.691630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caa70c-0db4-41f5-b77e-07a598b4aaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.692119 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caa70c-0db4-41f5-b77e-07a598b4aaa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.693061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caa70c-0db4-41f5-b77e-07a598b4aaa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.695887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caa70c-0db4-41f5-b77e-07a598b4aaa2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.713449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 849/1711] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:24.714700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caa70c-0db4-41f5-b77e-07a598b4aaa2 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.716809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-96caa70c-0db4-41f5-b77e-07a598b4aaa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:24.730753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caa70c-0db4-41f5-b77e-07a598b4aaa2 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.735606 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a33b99-f0b8-4a59-8aaa-833fdc82bce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.736538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a33b99-f0b8-4a59-8aaa-833fdc82bce4 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.736818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a33b99-f0b8-4a59-8aaa-833fdc82bce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.737068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c280a-c96a-420e-ba9b-41896c1174c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.737179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a33b99-f0b8-4a59-8aaa-833fdc82bce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.739077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c280a-c96a-420e-ba9b-41896c1174c8 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.748089 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a33b99-f0b8-4a59-8aaa-833fdc82bce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.749590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a33b99-f0b8-4a59-8aaa-833fdc82bce4 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.751437 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 863/1712] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:24.752365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c280a-c96a-420e-ba9b-41896c1174c8 service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.770948 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 850/1713] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:24.780894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c280a-c96a-420e-ba9b-41896c1174c8 service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.786554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772b772-fa95-4e48-96fd-1e1827e20c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.787504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772b772-fa95-4e48-96fd-1e1827e20c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.788007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772b772-fa95-4e48-96fd-1e1827e20c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.788465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772b772-fa95-4e48-96fd-1e1827e20c2f admin admin] PATH_INFO: `/v3/projects/e46513de29f948cbbe7217f39a8c6718/users/47c52e0a707f4b73bd02902565a627b8/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.812534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772b772-fa95-4e48-96fd-1e1827e20c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46513de29f948cbbe7217f39a8c6718, user_id=47c52e0a707f4b73bd02902565a627b8, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:24.816144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772b772-fa95-4e48-96fd-1e1827e20c2f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:24.839086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 864/1714] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:24 2022] PUT /identity/v3/projects/e46513de29f948cbbe7217f39a8c6718/users/47c52e0a707f4b73bd02902565a627b8/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:24.850250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21f06-a0b1-45bd-ae38-79e284684054 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.850715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21f06-a0b1-45bd-ae38-79e284684054 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.850899 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21f06-a0b1-45bd-ae38-79e284684054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.882588 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772b772-fa95-4e48-96fd-1e1827e20c2f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:24.902121 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-c5a21f06-a0b1-45bd-ae38-79e284684054 None None] MFA Rules not processed for user `47c52e0a707f4b73bd02902565a627b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:24.904165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c5a21f06-a0b1-45bd-ae38-79e284684054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:24.965131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772b772-fa95-4e48-96fd-1e1827e20c2f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:24.970605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ac2a8-8f94-465b-8894-1e21f7672543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:24.971327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ac2a8-8f94-465b-8894-1e21f7672543 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:24.971708 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ac2a8-8f94-465b-8894-1e21f7672543 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:24.972064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ac2a8-8f94-465b-8894-1e21f7672543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:24.974904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-808ac2a8-8f94-465b-8894-1e21f7672543 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:24.980067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:25.009151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 851/1715] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:25.010609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a21f06-a0b1-45bd-ae38-79e284684054 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:25.058668 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ac2a8-8f94-465b-8894-1e21f7672543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:25.060323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ac2a8-8f94-465b-8894-1e21f7672543 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:25.081153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 865/1716] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:41:25.139077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683847fe-21b6-42c7-87f1-d40840dde73f None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.139370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683847fe-21b6-42c7-87f1-d40840dde73f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.139649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683847fe-21b6-42c7-87f1-d40840dde73f None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.141353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 866/1717] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:41:25.152976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a21f06-a0b1-45bd-ae38-79e284684054 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:25.154016 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5243c-461a-4c56-81e0-96f164b9534c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.154306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5243c-461a-4c56-81e0-96f164b9534c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.154592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5243c-461a-4c56-81e0-96f164b9534c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.171554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab203e-6530-448e-adef-d78edad1b56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:25.172191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab203e-6530-448e-adef-d78edad1b56d admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.172468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab203e-6530-448e-adef-d78edad1b56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.172733 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab203e-6530-448e-adef-d78edad1b56d admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.173565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab203e-6530-448e-adef-d78edad1b56d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:25.180588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab203e-6530-448e-adef-d78edad1b56d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:25.212059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-edc5243c-461a-4c56-81e0-96f164b9534c None None] MFA Rules not processed for user `99ec4a4276f345c2998723e554ab9608`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:25.213906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-edc5243c-461a-4c56-81e0-96f164b9534c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:25.258565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 852/1718] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:25 2022] POST /identity/v3/projects => generated 385 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:25.263166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab203e-6530-448e-adef-d78edad1b56d admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:25.287597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:25.392975 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 867/1719] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:41:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:25.394121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5243c-461a-4c56-81e0-96f164b9534c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:25.395647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-edc5243c-461a-4c56-81e0-96f164b9534c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:25.403506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab203e-6530-448e-adef-d78edad1b56d admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:25.409352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5243c-461a-4c56-81e0-96f164b9534c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:25.409553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7bd09-7862-42da-be50-39df8785a7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:25.410235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7bd09-7862-42da-be50-39df8785a7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.410514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7bd09-7862-42da-be50-39df8785a7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.410881 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7bd09-7862-42da-be50-39df8785a7a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.411566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7bd09-7862-42da-be50-39df8785a7a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:25.413799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7bd09-7862-42da-be50-39df8785a7a5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:25.416631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:25.416631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.416631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.416631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] PATH_INFO: `/v3/users/eb41f09835414ed1a585f2485bb442cc` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.417922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb41f09835414ed1a585f2485bb442cc)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:25.419534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:25.443049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 853/1720] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:25 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:25.443049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7bd09-7862-42da-be50-39df8785a7a5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:25.445035 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e9d7bd09-7862-42da-be50-39df8785a7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:25.473217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb41f09835414ed1a585f2485bb442cc'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:25.488100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb41f09835414ed1a585f2485bb442cc'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:25.488945 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb41f09835414ed1a585f2485bb442cc'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:25.505107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb41f09835414ed1a585f2485bb442cc'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:25.506047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb41f09835414ed1a585f2485bb442cc'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:25.531260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 868/1721] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:25 2022] DELETE /identity/v3/users/eb41f09835414ed1a585f2485bb442cc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:25.532139 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:25.533501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:25.567684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7bd09-7862-42da-be50-39df8785a7a5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:25.572139 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:25.665519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738de071-64b6-4552-9f8a-656fff5221f9 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:25.672115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cdb51-0549-4050-9288-0d4fbfad5203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:25.672874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cdb51-0549-4050-9288-0d4fbfad5203 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.673182 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cdb51-0549-4050-9288-0d4fbfad5203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.673499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cdb51-0549-4050-9288-0d4fbfad5203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.674204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cdb51-0549-4050-9288-0d4fbfad5203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:25.676049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cdb51-0549-4050-9288-0d4fbfad5203 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:25.677304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be740c1e-6e07-484b-92e7-b0b83f816b25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:25.678088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be740c1e-6e07-484b-92e7-b0b83f816b25 service placement] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.678477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be740c1e-6e07-484b-92e7-b0b83f816b25 service placement] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.679034 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be740c1e-6e07-484b-92e7-b0b83f816b25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.680695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-be740c1e-6e07-484b-92e7-b0b83f816b25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:25.688271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 869/1722] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:25.692643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cdb51-0549-4050-9288-0d4fbfad5203 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:25.693770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be740c1e-6e07-484b-92e7-b0b83f816b25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:25.695383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be740c1e-6e07-484b-92e7-b0b83f816b25 service placement] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:25.702874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cdb51-0549-4050-9288-0d4fbfad5203 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:25.707514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 854/1723] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:25.707959 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e83e4-c2e8-477f-a2e0-9570b426ac22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:25.708755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e83e4-c2e8-477f-a2e0-9570b426ac22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.709298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e83e4-c2e8-477f-a2e0-9570b426ac22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.709639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e83e4-c2e8-477f-a2e0-9570b426ac22 admin admin] PATH_INFO: `/v3/projects/170bd2cba16d494d868f7c27a97ea948/users/e0a32100669643ba9324f97e38aee206/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.734234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e83e4-c2e8-477f-a2e0-9570b426ac22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170bd2cba16d494d868f7c27a97ea948, user_id=e0a32100669643ba9324f97e38aee206, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:25.736767 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e83e4-c2e8-477f-a2e0-9570b426ac22 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:25.751567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 870/1724] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:25 2022] PUT /identity/v3/projects/170bd2cba16d494d868f7c27a97ea948/users/e0a32100669643ba9324f97e38aee206/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:25.755795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be740c1e-6e07-484b-92e7-b0b83f816b25 service placement] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:25.766482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be740c1e-6e07-484b-92e7-b0b83f816b25 service placement] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:25.786439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e83e4-c2e8-477f-a2e0-9570b426ac22 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:25.788220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b96e83e4-c2e8-477f-a2e0-9570b426ac22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:25.803910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:25.846099 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfec60-aed4-4990-b7cf-68cc6d6df6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:25.846640 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfec60-aed4-4990-b7cf-68cc6d6df6bd admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.846926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfec60-aed4-4990-b7cf-68cc6d6df6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.847192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfec60-aed4-4990-b7cf-68cc6d6df6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.847962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfec60-aed4-4990-b7cf-68cc6d6df6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:25.849484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfec60-aed4-4990-b7cf-68cc6d6df6bd admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:25.860356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 855/1725] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:25.863397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e83e4-c2e8-477f-a2e0-9570b426ac22 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:25.864491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfec60-aed4-4990-b7cf-68cc6d6df6bd admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:25.868279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:25.868944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.869800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.870281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] PATH_INFO: `/v3/projects/25a8594d38034ae7805635a7b2de8ccb` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.872062 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a8594d38034ae7805635a7b2de8ccb)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:25.874003 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:25.875007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfec60-aed4-4990-b7cf-68cc6d6df6bd admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:25.880513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8979d-b72d-48ea-8e2d-9d25595db7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:25.881455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8979d-b72d-48ea-8e2d-9d25595db7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.881751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8979d-b72d-48ea-8e2d-9d25595db7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.882021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8979d-b72d-48ea-8e2d-9d25595db7c6 admin admin] PATH_INFO: `/v3/projects/170bd2cba16d494d868f7c27a97ea948/users/e0a32100669643ba9324f97e38aee206/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.884818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8979d-b72d-48ea-8e2d-9d25595db7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170bd2cba16d494d868f7c27a97ea948, user_id=e0a32100669643ba9324f97e38aee206, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:25.888351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8979d-b72d-48ea-8e2d-9d25595db7c6 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:25.900011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 856/1726] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:25 2022] PUT /identity/v3/projects/170bd2cba16d494d868f7c27a97ea948/users/e0a32100669643ba9324f97e38aee206/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:25.911906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed505ee-f5d5-41c3-b6bc-ebe7b2a1f513 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:25.912222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed505ee-f5d5-41c3-b6bc-ebe7b2a1f513 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:25.912464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed505ee-f5d5-41c3-b6bc-ebe7b2a1f513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:25.925718 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a8594d38034ae7805635a7b2de8ccb', 'request_id': 'req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:25.932261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a8594d38034ae7805635a7b2de8ccb', 'request_id': 'req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:25.941571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a8594d38034ae7805635a7b2de8ccb', 'request_id': 'req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:25.943712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4ed505ee-f5d5-41c3-b6bc-ebe7b2a1f513 None None] MFA Rules not processed for user `e0a32100669643ba9324f97e38aee206`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:25.944961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4ed505ee-f5d5-41c3-b6bc-ebe7b2a1f513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:25.954352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] The token cache is being invalidate because project 25a8594d38034ae7805635a7b2de8ccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:25.954648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:25.955406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:25.957057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 871/1727] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:25 2022] DELETE /identity/v3/projects/25a8594d38034ae7805635a7b2de8ccb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:25.960947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:25.962420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:26.020293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 857/1728] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:26.031573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d3ae5-4d04-43d9-bf5a-894ae94a9c68 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:26.033156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed505ee-f5d5-41c3-b6bc-ebe7b2a1f513 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:26.034582 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4ed505ee-f5d5-41c3-b6bc-ebe7b2a1f513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:26.036346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:26.037017 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:26.037288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:26.037564 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] PATH_INFO: `/v3/users/77e7142139174f29a2d4ee837c3195a0` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:26.038695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77e7142139174f29a2d4ee837c3195a0)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:26.040604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:26.041700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:26.086301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:26.089108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77e7142139174f29a2d4ee837c3195a0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:26.101919 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed505ee-f5d5-41c3-b6bc-ebe7b2a1f513 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:26.103827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77e7142139174f29a2d4ee837c3195a0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:26.104504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77e7142139174f29a2d4ee837c3195a0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:26.105642 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef49d9-40fc-46b5-92ba-f38db56ed997 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:26.106261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef49d9-40fc-46b5-92ba-f38db56ed997 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:26.106540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef49d9-40fc-46b5-92ba-f38db56ed997 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:26.106811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef49d9-40fc-46b5-92ba-f38db56ed997 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:26.108608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-62ef49d9-40fc-46b5-92ba-f38db56ed997 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:26.111486 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77e7142139174f29a2d4ee837c3195a0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:26.112137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77e7142139174f29a2d4ee837c3195a0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:26.126673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 872/1729] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:25 2022] DELETE /identity/v3/users/77e7142139174f29a2d4ee837c3195a0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:26.172441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef49d9-40fc-46b5-92ba-f38db56ed997 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:26.174125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef49d9-40fc-46b5-92ba-f38db56ed997 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:26.189538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 858/1730] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:41:26.335998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:26.340032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:26.436476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ca0f3-d15b-4cf0-b739-ec357949d290 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:26.441980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:26.442774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:26.443201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:26.443623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] PATH_INFO: `/v3/projects/3e3b62e3524a4b9d809ff96769d9edf8` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:26.445005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e3b62e3524a4b9d809ff96769d9edf8)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:26.447736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:26.466011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef49d9-40fc-46b5-92ba-f38db56ed997 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:26.469488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-62ef49d9-40fc-46b5-92ba-f38db56ed997 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:26.486118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef49d9-40fc-46b5-92ba-f38db56ed997 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:26.527202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e3b62e3524a4b9d809ff96769d9edf8', 'request_id': 'req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:26.534333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e3b62e3524a4b9d809ff96769d9edf8', 'request_id': 'req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:26.546639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e3b62e3524a4b9d809ff96769d9edf8', 'request_id': 'req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:26.561163 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74473397-fee0-4cda-af3f-7957f8965cbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:26.562083 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] The token cache is being invalidate because project 3e3b62e3524a4b9d809ff96769d9edf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:26.562224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74473397-fee0-4cda-af3f-7957f8965cbe service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:26.562378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:26.562645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74473397-fee0-4cda-af3f-7957f8965cbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:26.563037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74473397-fee0-4cda-af3f-7957f8965cbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:26.563147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b1bc8c62-23f6-4ce7-aa01-c55dbbb4a47a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:26.567938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 873/1731] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:26 2022] DELETE /identity/v3/projects/3e3b62e3524a4b9d809ff96769d9edf8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:26.568092 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-74473397-fee0-4cda-af3f-7957f8965cbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:26.569806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a312065-d058-42a1-be28-d2cbb78274ae None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:26.570234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a312065-d058-42a1-be28-d2cbb78274ae None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:26.570691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a312065-d058-42a1-be28-d2cbb78274ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:26.594415 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-4a312065-d058-42a1-be28-d2cbb78274ae None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:26.595836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4a312065-d058-42a1-be28-d2cbb78274ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:26.658814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:26.666459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74473397-fee0-4cda-af3f-7957f8965cbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:26.668589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74473397-fee0-4cda-af3f-7957f8965cbe service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:26.679009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 859/1732] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:26.687953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 874/1733] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:26.690954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceb5c6-1ed0-45d3-a77a-ee403731b105 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:26.691245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceb5c6-1ed0-45d3-a77a-ee403731b105 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:26.691516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceb5c6-1ed0-45d3-a77a-ee403731b105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:26.720946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-efceb5c6-1ed0-45d3-a77a-ee403731b105 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:26.722798 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-efceb5c6-1ed0-45d3-a77a-ee403731b105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:26.735559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 860/1734] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:26.740039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a312065-d058-42a1-be28-d2cbb78274ae None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:26.750671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a312065-d058-42a1-be28-d2cbb78274ae None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:26.756246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ff4d2-6cec-4fe5-b979-89e087a8da37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:26.757087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ff4d2-6cec-4fe5-b979-89e087a8da37 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:26.757466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ff4d2-6cec-4fe5-b979-89e087a8da37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:26.757861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ff4d2-6cec-4fe5-b979-89e087a8da37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:26.758613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ff4d2-6cec-4fe5-b979-89e087a8da37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:26.759022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ff4d2-6cec-4fe5-b979-89e087a8da37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:26.761488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ff4d2-6cec-4fe5-b979-89e087a8da37 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:26.788490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 875/1735] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:26.793110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceb5c6-1ed0-45d3-a77a-ee403731b105 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:26.804401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceb5c6-1ed0-45d3-a77a-ee403731b105 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:26.811930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5f3f3-b816-4f65-affb-e23fc8c7fdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:26.813016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5f3f3-b816-4f65-affb-e23fc8c7fdf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:26.813503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5f3f3-b816-4f65-affb-e23fc8c7fdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:26.813976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5f3f3-b816-4f65-affb-e23fc8c7fdf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:26.814895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5f3f3-b816-4f65-affb-e23fc8c7fdf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:26.818079 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5f3f3-b816-4f65-affb-e23fc8c7fdf4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:26.825333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:26.880533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 861/1736] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:26 2022] POST /identity/v3/projects => generated 359 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:26.881908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ff4d2-6cec-4fe5-b979-89e087a8da37 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:26.894410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ff4d2-6cec-4fe5-b979-89e087a8da37 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:26.966286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e57920-4cb4-4c06-bbe3-e0924a74fff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:26.967001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57920-4cb4-4c06-bbe3-e0924a74fff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:26.967183 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57920-4cb4-4c06-bbe3-e0924a74fff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:26.967458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57920-4cb4-4c06-bbe3-e0924a74fff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:26.968110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e57920-4cb4-4c06-bbe3-e0924a74fff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:26.970153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e57920-4cb4-4c06-bbe3-e0924a74fff6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:26.995171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 876/1737] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:26 2022] POST /identity/v3/users => generated 280 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:26.999295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5f3f3-b816-4f65-affb-e23fc8c7fdf4 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:27.013276 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5f3f3-b816-4f65-affb-e23fc8c7fdf4 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:27.034995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a086c-4db0-4003-a20f-1529d49ca10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:27.035667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a086c-4db0-4003-a20f-1529d49ca10b admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:27.036054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a086c-4db0-4003-a20f-1529d49ca10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:27.036191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a086c-4db0-4003-a20f-1529d49ca10b admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:27.036946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a086c-4db0-4003-a20f-1529d49ca10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:27.038598 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a086c-4db0-4003-a20f-1529d49ca10b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:27.049527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 862/1738] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:27.053735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e57920-4cb4-4c06-bbe3-e0924a74fff6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:27.064670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e57920-4cb4-4c06-bbe3-e0924a74fff6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:27.069628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcca50d-f701-4fb5-b098-b242dca3154e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:27.070232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcca50d-f701-4fb5-b098-b242dca3154e admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:27.070509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcca50d-f701-4fb5-b098-b242dca3154e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:27.070780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcca50d-f701-4fb5-b098-b242dca3154e admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:27.071422 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcca50d-f701-4fb5-b098-b242dca3154e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:27.073658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcca50d-f701-4fb5-b098-b242dca3154e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:27.084474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 877/1739] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:27.089068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a086c-4db0-4003-a20f-1529d49ca10b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:27.099275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a086c-4db0-4003-a20f-1529d49ca10b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:27.104534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0810b53-fdf7-413e-8d89-e5f324feac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:27.105244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0810b53-fdf7-413e-8d89-e5f324feac10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:27.105519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0810b53-fdf7-413e-8d89-e5f324feac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:27.105792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0810b53-fdf7-413e-8d89-e5f324feac10 admin admin] PATH_INFO: `/v3/projects/99061890273d4e819961f85bb67bd413/users/6ff6bad4834b4eafb14d2315235fed82/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:27.122058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0810b53-fdf7-413e-8d89-e5f324feac10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99061890273d4e819961f85bb67bd413, user_id=6ff6bad4834b4eafb14d2315235fed82, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:27.124825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0810b53-fdf7-413e-8d89-e5f324feac10 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:27.138294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 863/1740] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:27 2022] PUT /identity/v3/projects/99061890273d4e819961f85bb67bd413/users/6ff6bad4834b4eafb14d2315235fed82/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:27.148810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbf5e-9dba-4f19-a8ba-a31ae04f75d7 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:27.149123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbf5e-9dba-4f19-a8ba-a31ae04f75d7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:27.149407 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbf5e-9dba-4f19-a8ba-a31ae04f75d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:27.182957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-52bcbf5e-9dba-4f19-a8ba-a31ae04f75d7 None None] MFA Rules not processed for user `6ff6bad4834b4eafb14d2315235fed82`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:27.184364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-52bcbf5e-9dba-4f19-a8ba-a31ae04f75d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:27.257534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 878/1741] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:27.301098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0810b53-fdf7-413e-8d89-e5f324feac10 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:27.301098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b0810b53-fdf7-413e-8d89-e5f324feac10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:27.354383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0810b53-fdf7-413e-8d89-e5f324feac10 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:27.369171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09880990-2f85-443f-9a72-b11c927c3e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:27.369171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09880990-2f85-443f-9a72-b11c927c3e50 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:27.369171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09880990-2f85-443f-9a72-b11c927c3e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:27.369171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09880990-2f85-443f-9a72-b11c927c3e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:27.369602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09880990-2f85-443f-9a72-b11c927c3e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:27.369953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09880990-2f85-443f-9a72-b11c927c3e50 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:27.386099 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 864/1742] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:27.402933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbf5e-9dba-4f19-a8ba-a31ae04f75d7 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:27.427052 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09880990-2f85-443f-9a72-b11c927c3e50 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:27.451490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09880990-2f85-443f-9a72-b11c927c3e50 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:27.455344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a540d-54c8-4924-97a7-8d334fd55c85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:27.455990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a540d-54c8-4924-97a7-8d334fd55c85 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:27.456264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a540d-54c8-4924-97a7-8d334fd55c85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:27.456538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a540d-54c8-4924-97a7-8d334fd55c85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:27.458381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-584a540d-54c8-4924-97a7-8d334fd55c85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:27.486426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:27.509379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbf5e-9dba-4f19-a8ba-a31ae04f75d7 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:27.515007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:27.515376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:27.515655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:27.516117 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] PATH_INFO: `/v3/users/6ff6bad4834b4eafb14d2315235fed82` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:27.517419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff6bad4834b4eafb14d2315235fed82)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:27.519398 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:27.547258 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a540d-54c8-4924-97a7-8d334fd55c85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:27.551044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a540d-54c8-4924-97a7-8d334fd55c85 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:27.565382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 865/1743] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:27.582686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff6bad4834b4eafb14d2315235fed82'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:27.600367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff6bad4834b4eafb14d2315235fed82'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:27.601167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff6bad4834b4eafb14d2315235fed82'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:27.613089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff6bad4834b4eafb14d2315235fed82'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:27.613859 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff6bad4834b4eafb14d2315235fed82'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:27.631496 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 879/1744] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:27 2022] DELETE /identity/v3/users/6ff6bad4834b4eafb14d2315235fed82 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:27.636713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73729f9c-9994-4ad1-9a91-90e0fff24382 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:27.637023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73729f9c-9994-4ad1-9a91-90e0fff24382 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:27.637367 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73729f9c-9994-4ad1-9a91-90e0fff24382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:27.666394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-73729f9c-9994-4ad1-9a91-90e0fff24382 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:27.667831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-73729f9c-9994-4ad1-9a91-90e0fff24382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:27.720812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:27.741359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 866/1745] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:27.752263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:27.753911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:27.823364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f855a-5c77-493f-9a2b-9c4fb73fc440 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:27.827166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d4a53-4e8f-4202-8c4f-64e21145e087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:27.828048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d4a53-4e8f-4202-8c4f-64e21145e087 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:27.828332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d4a53-4e8f-4202-8c4f-64e21145e087 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:27.828712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d4a53-4e8f-4202-8c4f-64e21145e087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:27.840444 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d4a53-4e8f-4202-8c4f-64e21145e087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:27.842177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d4a53-4e8f-4202-8c4f-64e21145e087 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:27.856491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 880/1746] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:41:28.256211 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73729f9c-9994-4ad1-9a91-90e0fff24382 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:28.257398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-73729f9c-9994-4ad1-9a91-90e0fff24382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:28.274676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73729f9c-9994-4ad1-9a91-90e0fff24382 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:28.281261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:28.281931 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:28.282179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:28.282457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] PATH_INFO: `/v3/projects/99061890273d4e819961f85bb67bd413` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:28.286841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99061890273d4e819961f85bb67bd413)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:28.288863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:28.345457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99061890273d4e819961f85bb67bd413', 'request_id': 'req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:28.351393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99061890273d4e819961f85bb67bd413', 'request_id': 'req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:28.358943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99061890273d4e819961f85bb67bd413', 'request_id': 'req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:28.371093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] The token cache is being invalidate because project 99061890273d4e819961f85bb67bd413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:28.371379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:28.372017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-21dfdf80-0ff2-457b-bc7b-1b2fe2fbe840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:28.373552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 867/1747] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:28 2022] DELETE /identity/v3/projects/99061890273d4e819961f85bb67bd413 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:28.378554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba3ba7-5ca2-478e-a71f-49516590d879 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:28.379011 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba3ba7-5ca2-478e-a71f-49516590d879 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:28.379112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba3ba7-5ca2-478e-a71f-49516590d879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:28.399658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6aba3ba7-5ca2-478e-a71f-49516590d879 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:28.401136 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6aba3ba7-5ca2-478e-a71f-49516590d879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:28.467496 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 881/1748] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:28.477472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc7507-3cb3-4353-bef3-51ce8099a957 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:28.478014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc7507-3cb3-4353-bef3-51ce8099a957 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:28.478014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc7507-3cb3-4353-bef3-51ce8099a957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:28.510808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a8cc7507-3cb3-4353-bef3-51ce8099a957 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:28.512057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a8cc7507-3cb3-4353-bef3-51ce8099a957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:28.529800 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 868/1749] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:28.534456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba3ba7-5ca2-478e-a71f-49516590d879 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:28.546140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba3ba7-5ca2-478e-a71f-49516590d879 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:28.552914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f3ff0-49ba-41f7-9e03-383979f213e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:28.553965 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f3ff0-49ba-41f7-9e03-383979f213e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:28.554395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f3ff0-49ba-41f7-9e03-383979f213e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:28.554896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f3ff0-49ba-41f7-9e03-383979f213e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:28.555704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f3ff0-49ba-41f7-9e03-383979f213e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:28.556160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f3ff0-49ba-41f7-9e03-383979f213e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:28.559384 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f3ff0-49ba-41f7-9e03-383979f213e1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:28.562091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:28.577026 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 882/1750] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:41:28.581048 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc7507-3cb3-4353-bef3-51ce8099a957 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:28.594370 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc7507-3cb3-4353-bef3-51ce8099a957 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:28.601476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52a467-5f8d-4975-b87d-39a81a610f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:28.602415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52a467-5f8d-4975-b87d-39a81a610f6b admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:28.602782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52a467-5f8d-4975-b87d-39a81a610f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:28.605004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52a467-5f8d-4975-b87d-39a81a610f6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:28.605702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52a467-5f8d-4975-b87d-39a81a610f6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:28.611773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52a467-5f8d-4975-b87d-39a81a610f6b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:28.649697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 869/1751] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:28 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:28.650823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f3ff0-49ba-41f7-9e03-383979f213e1 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:28.661857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f3ff0-49ba-41f7-9e03-383979f213e1 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:28.742198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad6e44-df7d-4b05-a145-e9bfc6296da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:28.745841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad6e44-df7d-4b05-a145-e9bfc6296da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:28.746130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad6e44-df7d-4b05-a145-e9bfc6296da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:28.746409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad6e44-df7d-4b05-a145-e9bfc6296da4 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:28.747181 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad6e44-df7d-4b05-a145-e9bfc6296da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:28.758608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad6e44-df7d-4b05-a145-e9bfc6296da4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:28.793260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 883/1752] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:28 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:41:28.797243 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52a467-5f8d-4975-b87d-39a81a610f6b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:28.811063 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52a467-5f8d-4975-b87d-39a81a610f6b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:28.816877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07875d3e-bb13-433d-bc6a-93688ce5991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:28.817717 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875d3e-bb13-433d-bc6a-93688ce5991b admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:28.818300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875d3e-bb13-433d-bc6a-93688ce5991b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:28.818758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875d3e-bb13-433d-bc6a-93688ce5991b admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:28.819790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07875d3e-bb13-433d-bc6a-93688ce5991b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:28.821810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07875d3e-bb13-433d-bc6a-93688ce5991b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:28.839132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 870/1753] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:41:28.844379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad6e44-df7d-4b05-a145-e9bfc6296da4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:28.856377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad6e44-df7d-4b05-a145-e9bfc6296da4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:28.862824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc2bd4-9f99-4671-b466-6f53b014f028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:28.863450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc2bd4-9f99-4671-b466-6f53b014f028 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:28.863725 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc2bd4-9f99-4671-b466-6f53b014f028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:28.864021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc2bd4-9f99-4671-b466-6f53b014f028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:28.865536 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc2bd4-9f99-4671-b466-6f53b014f028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:28.867699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc2bd4-9f99-4671-b466-6f53b014f028 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:28.880455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 884/1754] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:28.886960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07875d3e-bb13-433d-bc6a-93688ce5991b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:28.906631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07875d3e-bb13-433d-bc6a-93688ce5991b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:28.913436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66976e42-08df-4ac4-9526-ff513c7f05aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:28.914380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66976e42-08df-4ac4-9526-ff513c7f05aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:28.914819 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66976e42-08df-4ac4-9526-ff513c7f05aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:28.915349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66976e42-08df-4ac4-9526-ff513c7f05aa admin admin] PATH_INFO: `/v3/projects/047b4fe1eab744d9b871985c30e76b0d/users/fcb7dccab5ec46348c87ad4889120109/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:28.937433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66976e42-08df-4ac4-9526-ff513c7f05aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=047b4fe1eab744d9b871985c30e76b0d, user_id=fcb7dccab5ec46348c87ad4889120109, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:28.940429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66976e42-08df-4ac4-9526-ff513c7f05aa admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:28.986128 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18b502-9fe7-4248-8792-267b9a8e4e7d None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:28.986611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18b502-9fe7-4248-8792-267b9a8e4e7d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:28.987271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18b502-9fe7-4248-8792-267b9a8e4e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:28.987624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 871/1755] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:28 2022] PUT /identity/v3/projects/047b4fe1eab744d9b871985c30e76b0d/users/fcb7dccab5ec46348c87ad4889120109/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:29.009655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1d18b502-9fe7-4248-8792-267b9a8e4e7d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:29.011102 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1d18b502-9fe7-4248-8792-267b9a8e4e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:29.133064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 885/1756] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:29.161581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66976e42-08df-4ac4-9526-ff513c7f05aa admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:29.175595 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-66976e42-08df-4ac4-9526-ff513c7f05aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:29.267196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66976e42-08df-4ac4-9526-ff513c7f05aa admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:29.271094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eefe9-48c5-41d1-a41b-3a7a3aa7ee0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:29.271784 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eefe9-48c5-41d1-a41b-3a7a3aa7ee0b service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:29.271973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eefe9-48c5-41d1-a41b-3a7a3aa7ee0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:29.272247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eefe9-48c5-41d1-a41b-3a7a3aa7ee0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:29.283874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1eefe9-48c5-41d1-a41b-3a7a3aa7ee0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:29.286138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1eefe9-48c5-41d1-a41b-3a7a3aa7ee0b service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:29.309787 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 872/1757] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:30.277967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0633afc-77ff-4e4c-8e4e-463d0950dad8 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:30.278490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0633afc-77ff-4e4c-8e4e-463d0950dad8 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:30.278709 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0633afc-77ff-4e4c-8e4e-463d0950dad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:30.319838 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c0633afc-77ff-4e4c-8e4e-463d0950dad8 None None] MFA Rules not processed for user `fcb7dccab5ec46348c87ad4889120109`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:30.321255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c0633afc-77ff-4e4c-8e4e-463d0950dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:30.423785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 886/1758] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:30.468705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eefe9-48c5-41d1-a41b-3a7a3aa7ee0b service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:30.470692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9a1eefe9-48c5-41d1-a41b-3a7a3aa7ee0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:30.483170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:30.554968 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eefe9-48c5-41d1-a41b-3a7a3aa7ee0b service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:30.566438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03c2d8-c840-4344-a221-2c9cc4ff34bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:30.566983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03c2d8-c840-4344-a221-2c9cc4ff34bd service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:30.567256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03c2d8-c840-4344-a221-2c9cc4ff34bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:30.567522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03c2d8-c840-4344-a221-2c9cc4ff34bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:30.580449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03c2d8-c840-4344-a221-2c9cc4ff34bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:30.582182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03c2d8-c840-4344-a221-2c9cc4ff34bd service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:30.593869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 873/1759] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:30.749968 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0633afc-77ff-4e4c-8e4e-463d0950dad8 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:30.751740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c0633afc-77ff-4e4c-8e4e-463d0950dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:30.763540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:30.769272 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0633afc-77ff-4e4c-8e4e-463d0950dad8 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:30.775744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:30.776502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:30.776837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:30.777208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] PATH_INFO: `/v3/users/34557b30965747c496c294e2474039c2` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:30.778401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34557b30965747c496c294e2474039c2)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:30.780699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:30.839290 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34557b30965747c496c294e2474039c2'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:30.860623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34557b30965747c496c294e2474039c2'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:30.861440 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34557b30965747c496c294e2474039c2'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:30.864316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03c2d8-c840-4344-a221-2c9cc4ff34bd service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:30.866006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0e03c2d8-c840-4344-a221-2c9cc4ff34bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:30.869761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34557b30965747c496c294e2474039c2'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:30.870639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-51ae0c16-3de3-4bd5-ac90-a61856e2b153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34557b30965747c496c294e2474039c2'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:30.880531 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03c2d8-c840-4344-a221-2c9cc4ff34bd service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:30.884905 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839307ba-0ef6-4683-9154-d50560d4660f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:30.885756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839307ba-0ef6-4683-9154-d50560d4660f service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:30.886261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839307ba-0ef6-4683-9154-d50560d4660f service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:30.886696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839307ba-0ef6-4683-9154-d50560d4660f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:30.889016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-839307ba-0ef6-4683-9154-d50560d4660f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:30.891847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 887/1760] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:30 2022] DELETE /identity/v3/users/34557b30965747c496c294e2474039c2 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:30.909994 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1661f8-4f22-4842-b97a-1d4dd5c807df None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:30.911127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1661f8-4f22-4842-b97a-1d4dd5c807df None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:30.913285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1661f8-4f22-4842-b97a-1d4dd5c807df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:30.938151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-3a1661f8-4f22-4842-b97a-1d4dd5c807df None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:30.939794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3a1661f8-4f22-4842-b97a-1d4dd5c807df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:30.984949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839307ba-0ef6-4683-9154-d50560d4660f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:30.986694 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839307ba-0ef6-4683-9154-d50560d4660f service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:31.000042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 874/1761] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:41:31.030561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 888/1762] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:31.041306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839307ba-0ef6-4683-9154-d50560d4660f service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:31.043694 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-839307ba-0ef6-4683-9154-d50560d4660f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:31.056021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839307ba-0ef6-4683-9154-d50560d4660f service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:31.108292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd817a0d-8eba-4bc5-b2f6-de922b885c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:31.108930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd817a0d-8eba-4bc5-b2f6-de922b885c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:31.109329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd817a0d-8eba-4bc5-b2f6-de922b885c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:31.109621 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd817a0d-8eba-4bc5-b2f6-de922b885c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:31.118880 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd817a0d-8eba-4bc5-b2f6-de922b885c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:31.121000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd817a0d-8eba-4bc5-b2f6-de922b885c56 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:31.132243 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 875/1763] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:31.731997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1661f8-4f22-4842-b97a-1d4dd5c807df None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:31.734157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3a1661f8-4f22-4842-b97a-1d4dd5c807df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:31.746759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1661f8-4f22-4842-b97a-1d4dd5c807df None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:31.752645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:31.753358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:31.753636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:31.754064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] PATH_INFO: `/v3/projects/7d9a90f4e79d4b8eaf2f1da23ca9c5db` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:31.755383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9a90f4e79d4b8eaf2f1da23ca9c5db)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:31.757580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:31.808664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9a90f4e79d4b8eaf2f1da23ca9c5db', 'request_id': 'req-b7d0c95f-fb3f-480a-8341-181fa836395d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:31.816643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9a90f4e79d4b8eaf2f1da23ca9c5db', 'request_id': 'req-b7d0c95f-fb3f-480a-8341-181fa836395d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:31.827345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9a90f4e79d4b8eaf2f1da23ca9c5db', 'request_id': 'req-b7d0c95f-fb3f-480a-8341-181fa836395d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:31.842286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:31.847950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] The token cache is being invalidate because project 7d9a90f4e79d4b8eaf2f1da23ca9c5db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:31.849104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:31.850709 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:31.856629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 889/1764] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:31 2022] DELETE /identity/v3/projects/7d9a90f4e79d4b8eaf2f1da23ca9c5db => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:31.857730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd817a0d-8eba-4bc5-b2f6-de922b885c56 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:31.869775 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd817a0d-8eba-4bc5-b2f6-de922b885c56 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:31.888290 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:31.889868 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:31.942622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:31.966340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:31.967064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:31.967346 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:31.967611 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] PATH_INFO: `/v3/users/e78383ff34fc48f8b1d3843fdde1533b` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:31.969148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e78383ff34fc48f8b1d3843fdde1533b)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:31.972113 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:32.010138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0c95f-fb3f-480a-8341-181fa836395d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:32.025988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:32.026660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:32.026930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:32.027308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] PATH_INFO: `/v3/users/af5bfb7a152f432198061c3538a71276` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:32.031823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5bfb7a152f432198061c3538a71276)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:32.044941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e78383ff34fc48f8b1d3843fdde1533b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.047201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:32.056095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e78383ff34fc48f8b1d3843fdde1533b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.062066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e78383ff34fc48f8b1d3843fdde1533b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.088039 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e78383ff34fc48f8b1d3843fdde1533b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.102311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-945e8b32-fb9e-418a-9c64-52c4834dd554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e78383ff34fc48f8b1d3843fdde1533b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.111499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 876/1765] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:31 2022] DELETE /identity/v3/users/e78383ff34fc48f8b1d3843fdde1533b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:32.114555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da015f8a-518e-4e6b-9f0a-c5dfbf122483 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:32.114555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da015f8a-518e-4e6b-9f0a-c5dfbf122483 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:32.114555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da015f8a-518e-4e6b-9f0a-c5dfbf122483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:32.128770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5bfb7a152f432198061c3538a71276'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.150741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5bfb7a152f432198061c3538a71276'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.152933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5bfb7a152f432198061c3538a71276'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.156259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-da015f8a-518e-4e6b-9f0a-c5dfbf122483 None None] MFA Rules not processed for user `fcb7dccab5ec46348c87ad4889120109`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:32.157721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-da015f8a-518e-4e6b-9f0a-c5dfbf122483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:32.162451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5bfb7a152f432198061c3538a71276'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.163241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9ad34ce9-282c-450e-93cc-b2b5bccf7004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5bfb7a152f432198061c3538a71276'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.194711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 890/1766] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:31 2022] DELETE /identity/v3/users/af5bfb7a152f432198061c3538a71276 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:32.200045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f90fbe-1188-431c-a1de-b28874498e90 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:32.201189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f90fbe-1188-431c-a1de-b28874498e90 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:32.201438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f90fbe-1188-431c-a1de-b28874498e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:32.221156 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-88f90fbe-1188-431c-a1de-b28874498e90 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:32.222344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-88f90fbe-1188-431c-a1de-b28874498e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:32.230662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 877/1767] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:32.304439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 891/1768] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:32.313008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da015f8a-518e-4e6b-9f0a-c5dfbf122483 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:32.315639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-da015f8a-518e-4e6b-9f0a-c5dfbf122483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:32.397774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da015f8a-518e-4e6b-9f0a-c5dfbf122483 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:32.439118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:32.488689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e7694-6b48-4052-905d-025e948736b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:32.489494 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e7694-6b48-4052-905d-025e948736b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:32.489926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e7694-6b48-4052-905d-025e948736b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:32.490295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e7694-6b48-4052-905d-025e948736b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:32.514453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e7694-6b48-4052-905d-025e948736b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:32.521951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e7694-6b48-4052-905d-025e948736b4 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:32.525740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f90fbe-1188-431c-a1de-b28874498e90 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:32.531862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-88f90fbe-1188-431c-a1de-b28874498e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:32.549887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 878/1769] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:41:32.570968 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f90fbe-1188-431c-a1de-b28874498e90 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:32.577124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:32.578030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:32.578407 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:32.578837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] PATH_INFO: `/v3/projects/8fd355c23e8c4519a4d44cfbb6462f7e` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:32.580598 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd355c23e8c4519a4d44cfbb6462f7e)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:32.583035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:32.604775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:32.678953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd355c23e8c4519a4d44cfbb6462f7e', 'request_id': 'req-060606d4-84a5-4e25-8f1e-4d5e04c87088'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.687158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd355c23e8c4519a4d44cfbb6462f7e', 'request_id': 'req-060606d4-84a5-4e25-8f1e-4d5e04c87088'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.703366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd355c23e8c4519a4d44cfbb6462f7e', 'request_id': 'req-060606d4-84a5-4e25-8f1e-4d5e04c87088'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.720869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] The token cache is being invalidate because project 8fd355c23e8c4519a4d44cfbb6462f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:32.721299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.722079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-060606d4-84a5-4e25-8f1e-4d5e04c87088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:32.729461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 892/1770] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:32 2022] DELETE /identity/v3/projects/8fd355c23e8c4519a4d44cfbb6462f7e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:32.731883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df04b7d-96f5-4b89-9be5-04724f540348 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:32.732297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df04b7d-96f5-4b89-9be5-04724f540348 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:32.732604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df04b7d-96f5-4b89-9be5-04724f540348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:32.791323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-5df04b7d-96f5-4b89-9be5-04724f540348 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:32.795772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5df04b7d-96f5-4b89-9be5-04724f540348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:32.954668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 879/1771] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:32.965692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaadcd9-9f16-4291-a242-324a9a07e075 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:32.966606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaadcd9-9f16-4291-a242-324a9a07e075 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:32.966606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaadcd9-9f16-4291-a242-324a9a07e075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:32.995476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-8eaadcd9-9f16-4291-a242-324a9a07e075 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:32.998063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8eaadcd9-9f16-4291-a242-324a9a07e075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:33.020503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 893/1772] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:33.024995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df04b7d-96f5-4b89-9be5-04724f540348 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:33.035431 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df04b7d-96f5-4b89-9be5-04724f540348 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:33.040709 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1734-a58c-4010-95de-2b358d057b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:33.041929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1734-a58c-4010-95de-2b358d057b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:33.042349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1734-a58c-4010-95de-2b358d057b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:33.042646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1734-a58c-4010-95de-2b358d057b92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:33.043288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b1734-a58c-4010-95de-2b358d057b92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:33.043590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b1734-a58c-4010-95de-2b358d057b92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:33.045234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b1734-a58c-4010-95de-2b358d057b92 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:33.065605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 880/1773] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:41:33.070321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaadcd9-9f16-4291-a242-324a9a07e075 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:33.084285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaadcd9-9f16-4291-a242-324a9a07e075 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:33.092725 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cee4f-d4b6-418b-8e2b-15d750be3625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:33.094116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cee4f-d4b6-418b-8e2b-15d750be3625 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:33.094441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cee4f-d4b6-418b-8e2b-15d750be3625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:33.095368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cee4f-d4b6-418b-8e2b-15d750be3625 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:33.097572 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cee4f-d4b6-418b-8e2b-15d750be3625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:33.100187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cee4f-d4b6-418b-8e2b-15d750be3625 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:33.140021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 894/1774] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:33 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:33.144463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1734-a58c-4010-95de-2b358d057b92 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:33.157712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1734-a58c-4010-95de-2b358d057b92 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:33.172673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cee4f-d4b6-418b-8e2b-15d750be3625 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:33.174497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2c7cee4f-d4b6-418b-8e2b-15d750be3625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:33.253061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997af0e-3338-4743-ad80-69312f136d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:33.253827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997af0e-3338-4743-ad80-69312f136d9f admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:33.254110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997af0e-3338-4743-ad80-69312f136d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:33.254991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997af0e-3338-4743-ad80-69312f136d9f admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:33.255646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997af0e-3338-4743-ad80-69312f136d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:33.258857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997af0e-3338-4743-ad80-69312f136d9f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:33.292891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 881/1775] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:33 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:33.296700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997af0e-3338-4743-ad80-69312f136d9f admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:33.310538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997af0e-3338-4743-ad80-69312f136d9f admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:33.321513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cee4f-d4b6-418b-8e2b-15d750be3625 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:33.327879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671e2a-8a75-416b-a5e7-693031d74bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:33.328527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671e2a-8a75-416b-a5e7-693031d74bce admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:33.328806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671e2a-8a75-416b-a5e7-693031d74bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:33.329145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671e2a-8a75-416b-a5e7-693031d74bce admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:33.330835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e671e2a-8a75-416b-a5e7-693031d74bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:33.333250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e671e2a-8a75-416b-a5e7-693031d74bce admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:33.339644 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:33.340650 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:33.340978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:33.341254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] PATH_INFO: `/v3/projects/e323a2101ba84b7697bb0546466021a6` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:33.342429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e323a2101ba84b7697bb0546466021a6)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:33.344349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:33.363710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 882/1776] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:33.367980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671e2a-8a75-416b-a5e7-693031d74bce admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:33.386276 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671e2a-8a75-416b-a5e7-693031d74bce admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:33.392477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b7e32-4354-44fe-9c1b-f30c69846ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:33.393393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b7e32-4354-44fe-9c1b-f30c69846ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:33.393563 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b7e32-4354-44fe-9c1b-f30c69846ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:33.394082 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b7e32-4354-44fe-9c1b-f30c69846ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:33.395795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b7e32-4354-44fe-9c1b-f30c69846ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:33.397934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b7e32-4354-44fe-9c1b-f30c69846ae2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:33.407426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e323a2101ba84b7697bb0546466021a6', 'request_id': 'req-b700d7a0-f3ec-4483-8f12-fccfc693047c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:33.413530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 883/1777] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:33.416674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e323a2101ba84b7697bb0546466021a6', 'request_id': 'req-b700d7a0-f3ec-4483-8f12-fccfc693047c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:33.418242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b7e32-4354-44fe-9c1b-f30c69846ae2 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:33.427793 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e323a2101ba84b7697bb0546466021a6', 'request_id': 'req-b700d7a0-f3ec-4483-8f12-fccfc693047c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:33.431221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b7e32-4354-44fe-9c1b-f30c69846ae2 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:33.442071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] The token cache is being invalidate because project e323a2101ba84b7697bb0546466021a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:33.442439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:33.443261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:33.444962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 895/1778] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:33 2022] DELETE /identity/v3/projects/e323a2101ba84b7697bb0546466021a6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:33.450214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:33.451803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:33.458734 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:33.471644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:33.510304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401155c-984c-4358-bd91-01f1b9acdbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:33.510869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401155c-984c-4358-bd91-01f1b9acdbba admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:33.511143 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401155c-984c-4358-bd91-01f1b9acdbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:33.511413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401155c-984c-4358-bd91-01f1b9acdbba admin admin] PATH_INFO: `/v3/projects/5ebf17b7731d40ae9f63af508cd3bc38/users/a196f0cc47c54c11bd618f7f33f8bdb2/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:33.528736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401155c-984c-4358-bd91-01f1b9acdbba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ebf17b7731d40ae9f63af508cd3bc38, user_id=a196f0cc47c54c11bd618f7f33f8bdb2, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:33.531433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401155c-984c-4358-bd91-01f1b9acdbba admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:33.533189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700d7a0-f3ec-4483-8f12-fccfc693047c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:33.538914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:33.539569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:33.539890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:33.540109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] PATH_INFO: `/v3/users/29e869c440164a82946db0eafb8c62e4` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:33.541335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29e869c440164a82946db0eafb8c62e4)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:33.543229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:33.550550 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 884/1779] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:33 2022] PUT /identity/v3/projects/5ebf17b7731d40ae9f63af508cd3bc38/users/a196f0cc47c54c11bd618f7f33f8bdb2/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:33.563379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ed4ca-84ff-4b7e-9b96-105cb44739b5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:33.564117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ed4ca-84ff-4b7e-9b96-105cb44739b5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:33.564394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ed4ca-84ff-4b7e-9b96-105cb44739b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:33.598460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a03ed4ca-84ff-4b7e-9b96-105cb44739b5 None None] MFA Rules not processed for user `a196f0cc47c54c11bd618f7f33f8bdb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:33.599848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a03ed4ca-84ff-4b7e-9b96-105cb44739b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:33.606939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29e869c440164a82946db0eafb8c62e4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:33.622190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29e869c440164a82946db0eafb8c62e4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:33.622960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29e869c440164a82946db0eafb8c62e4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:33.631096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29e869c440164a82946db0eafb8c62e4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:33.631791 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29e869c440164a82946db0eafb8c62e4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:33.646782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 896/1780] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:33 2022] DELETE /identity/v3/users/29e869c440164a82946db0eafb8c62e4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:33.680181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 885/1781] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:33.689558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:33.690987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:33.790996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6a445-c3ab-4475-a39c-d4311de383e6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:33.799034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e685-6fd5-47e3-8c96-0e960ec134ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d2a655f4d6f45039d641133e3b56592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:33.802567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e685-6fd5-47e3-8c96-0e960ec134ad service glance] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:33.802567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e685-6fd5-47e3-8c96-0e960ec134ad service glance] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:33.802567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e685-6fd5-47e3-8c96-0e960ec134ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:33.832379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:33.881980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935e685-6fd5-47e3-8c96-0e960ec134ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:33.883694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935e685-6fd5-47e3-8c96-0e960ec134ad service glance] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:33.898851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 897/1782] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:33.963432 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ed4ca-84ff-4b7e-9b96-105cb44739b5 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:33.964967 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a03ed4ca-84ff-4b7e-9b96-105cb44739b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:34.076963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ed4ca-84ff-4b7e-9b96-105cb44739b5 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:34.131444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:34.175666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:34.176324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:34.177874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:34.178812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] PATH_INFO: `/v3/projects/daf44851f7424d85aa3bcd66ab122aa9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:34.185199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf44851f7424d85aa3bcd66ab122aa9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:34.185199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:34.269552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf44851f7424d85aa3bcd66ab122aa9', 'request_id': 'req-5f18b804-e8a1-4617-b577-26aedd2935a5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:34.297688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf44851f7424d85aa3bcd66ab122aa9', 'request_id': 'req-5f18b804-e8a1-4617-b577-26aedd2935a5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:34.310960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf44851f7424d85aa3bcd66ab122aa9', 'request_id': 'req-5f18b804-e8a1-4617-b577-26aedd2935a5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:34.332191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] The token cache is being invalidate because project daf44851f7424d85aa3bcd66ab122aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:34.332516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:34.334510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:34.340943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 886/1783] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:33 2022] DELETE /identity/v3/projects/daf44851f7424d85aa3bcd66ab122aa9 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:34.343186 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e685-6fd5-47e3-8c96-0e960ec134ad service glance] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:34.360420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e685-6fd5-47e3-8c96-0e960ec134ad service glance] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:34.466198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:34.467086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:34.467445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:34.467812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] PATH_INFO: `/v3/users/226b9b89fec844219427b825eac1eed0` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:34.469089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226b9b89fec844219427b825eac1eed0)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:34.471229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:34.538844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226b9b89fec844219427b825eac1eed0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:34.562460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226b9b89fec844219427b825eac1eed0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:34.563205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226b9b89fec844219427b825eac1eed0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:34.570124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226b9b89fec844219427b825eac1eed0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:34.570777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-337ca3b6-c9d4-40c5-b524-5dc263b1f5c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226b9b89fec844219427b825eac1eed0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:34.587836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 898/1784] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:34 2022] DELETE /identity/v3/users/226b9b89fec844219427b825eac1eed0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:34.914967 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:34.917551 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:34.936750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18b804-e8a1-4617-b577-26aedd2935a5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:35.006916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:35.007637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.007885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.008168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] PATH_INFO: `/v3/projects/37f33517247e412eb0e83586644e433f` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.009387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f33517247e412eb0e83586644e433f)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:35.011297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:35.071419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f33517247e412eb0e83586644e433f', 'request_id': 'req-327a2a10-16c6-4b5d-a257-5439fc05ce06'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:35.078026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f33517247e412eb0e83586644e433f', 'request_id': 'req-327a2a10-16c6-4b5d-a257-5439fc05ce06'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:35.088529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f33517247e412eb0e83586644e433f', 'request_id': 'req-327a2a10-16c6-4b5d-a257-5439fc05ce06'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:35.103240 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] The token cache is being invalidate because project 37f33517247e412eb0e83586644e433f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:35.104169 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:35.104834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-327a2a10-16c6-4b5d-a257-5439fc05ce06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:35.106577 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 887/1785] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:34 2022] DELETE /identity/v3/projects/37f33517247e412eb0e83586644e433f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:35.111341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff41e5-86b9-491e-a693-293279d2b5e8 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.111762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff41e5-86b9-491e-a693-293279d2b5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.112142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff41e5-86b9-491e-a693-293279d2b5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.140533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-72ff41e5-86b9-491e-a693-293279d2b5e8 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:35.142395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-72ff41e5-86b9-491e-a693-293279d2b5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:35.197733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:35.233445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 899/1786] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:35.241518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c8255-1684-4259-b1d4-1bbd3a02fbcc None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.242259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c8255-1684-4259-b1d4-1bbd3a02fbcc None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.242512 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c8255-1684-4259-b1d4-1bbd3a02fbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.257603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:35.263764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-021c8255-1684-4259-b1d4-1bbd3a02fbcc None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:35.267116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-021c8255-1684-4259-b1d4-1bbd3a02fbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:35.279137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 888/1787] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:35.284694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff41e5-86b9-491e-a693-293279d2b5e8 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:35.295298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff41e5-86b9-491e-a693-293279d2b5e8 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:35.300027 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029248f3-c79b-4d05-9f52-c24f21041f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:35.300646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029248f3-c79b-4d05-9f52-c24f21041f2c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.300958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029248f3-c79b-4d05-9f52-c24f21041f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.301325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029248f3-c79b-4d05-9f52-c24f21041f2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.301949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029248f3-c79b-4d05-9f52-c24f21041f2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:35.302253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029248f3-c79b-4d05-9f52-c24f21041f2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:35.303849 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029248f3-c79b-4d05-9f52-c24f21041f2c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:35.316157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 900/1788] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:35.320681 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c8255-1684-4259-b1d4-1bbd3a02fbcc None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:35.330634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c8255-1684-4259-b1d4-1bbd3a02fbcc None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:35.335474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a9bc8-1c15-4d7d-8de1-5f2aad21a9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:35.336085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9bc8-1c15-4d7d-8de1-5f2aad21a9c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.336362 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9bc8-1c15-4d7d-8de1-5f2aad21a9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.336632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9bc8-1c15-4d7d-8de1-5f2aad21a9c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.337317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a9bc8-1c15-4d7d-8de1-5f2aad21a9c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:35.339888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a9bc8-1c15-4d7d-8de1-5f2aad21a9c5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:35.383675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029248f3-c79b-4d05-9f52-c24f21041f2c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:35.385414 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 889/1789] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:35 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:41:35.395072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029248f3-c79b-4d05-9f52-c24f21041f2c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:35.478537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75471ad2-ccb1-4a74-a0bb-af0de16cdaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:35.479264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75471ad2-ccb1-4a74-a0bb-af0de16cdaa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.479861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75471ad2-ccb1-4a74-a0bb-af0de16cdaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.480292 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75471ad2-ccb1-4a74-a0bb-af0de16cdaa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.481124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75471ad2-ccb1-4a74-a0bb-af0de16cdaa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:35.483592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75471ad2-ccb1-4a74-a0bb-af0de16cdaa0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:35.518618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 901/1790] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:35 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:35.519907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a9bc8-1c15-4d7d-8de1-5f2aad21a9c5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:35.533326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a9bc8-1c15-4d7d-8de1-5f2aad21a9c5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:35.539644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22107372-8b64-4841-92a3-e93cfc3a8794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:35.540410 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22107372-8b64-4841-92a3-e93cfc3a8794 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.541024 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22107372-8b64-4841-92a3-e93cfc3a8794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.541359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22107372-8b64-4841-92a3-e93cfc3a8794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.549742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22107372-8b64-4841-92a3-e93cfc3a8794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:35.549742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22107372-8b64-4841-92a3-e93cfc3a8794 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:35.570189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 890/1791] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:35.571067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75471ad2-ccb1-4a74-a0bb-af0de16cdaa0 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:35.584386 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75471ad2-ccb1-4a74-a0bb-af0de16cdaa0 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:35.597064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00723b-1675-4872-821a-a3f529d31360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:35.597738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00723b-1675-4872-821a-a3f529d31360 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.598087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00723b-1675-4872-821a-a3f529d31360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.598705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00723b-1675-4872-821a-a3f529d31360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.599483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00723b-1675-4872-821a-a3f529d31360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:35.601135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00723b-1675-4872-821a-a3f529d31360 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:35.615683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 902/1792] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:35.619560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22107372-8b64-4841-92a3-e93cfc3a8794 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:35.632990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22107372-8b64-4841-92a3-e93cfc3a8794 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:35.642341 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d59f5-59d2-4d9d-b88a-a839b25831a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:35.643027 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d59f5-59d2-4d9d-b88a-a839b25831a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.643335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d59f5-59d2-4d9d-b88a-a839b25831a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.644153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d59f5-59d2-4d9d-b88a-a839b25831a7 admin admin] PATH_INFO: `/v3/projects/4fb846ed3b9141c694927b06b387b433/users/f286fcc4b509485dbb958873a2207592/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.665775 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d59f5-59d2-4d9d-b88a-a839b25831a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb846ed3b9141c694927b06b387b433, user_id=f286fcc4b509485dbb958873a2207592, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:35.669751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d59f5-59d2-4d9d-b88a-a839b25831a7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:35.697829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 891/1793] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:35 2022] PUT /identity/v3/projects/4fb846ed3b9141c694927b06b387b433/users/f286fcc4b509485dbb958873a2207592/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:35.710755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e9cc4-e23e-437a-90e8-23fe4bb2602c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.711230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e9cc4-e23e-437a-90e8-23fe4bb2602c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.711712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e9cc4-e23e-437a-90e8-23fe4bb2602c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.766554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-829e9cc4-e23e-437a-90e8-23fe4bb2602c None None] MFA Rules not processed for user `f286fcc4b509485dbb958873a2207592`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:35.768595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-829e9cc4-e23e-437a-90e8-23fe4bb2602c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:35.889896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 903/1794] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:35.902474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d59f5-59d2-4d9d-b88a-a839b25831a7 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:35.904192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-316d59f5-59d2-4d9d-b88a-a839b25831a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:35.978682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d59f5-59d2-4d9d-b88a-a839b25831a7 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:35.983988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf061ea-8c1a-450e-8b3b-473720db3a62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:35.985620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf061ea-8c1a-450e-8b3b-473720db3a62 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:35.985976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf061ea-8c1a-450e-8b3b-473720db3a62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:35.986374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf061ea-8c1a-450e-8b3b-473720db3a62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:35.999632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf061ea-8c1a-450e-8b3b-473720db3a62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:36.002215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf061ea-8c1a-450e-8b3b-473720db3a62 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:36.017182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 892/1795] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:36.591109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e9cc4-e23e-437a-90e8-23fe4bb2602c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:36.593855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-829e9cc4-e23e-437a-90e8-23fe4bb2602c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:36.646138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:36.687549 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e9cc4-e23e-437a-90e8-23fe4bb2602c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:36.692217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:36.692906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:36.693198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:36.693469 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] PATH_INFO: `/v3/users/47c52e0a707f4b73bd02902565a627b8` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:36.695848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c52e0a707f4b73bd02902565a627b8)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:36.697691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:36.764634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c52e0a707f4b73bd02902565a627b8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:36.788223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c52e0a707f4b73bd02902565a627b8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:36.789151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c52e0a707f4b73bd02902565a627b8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:36.803131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c52e0a707f4b73bd02902565a627b8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:36.804502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c52e0a707f4b73bd02902565a627b8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:36.829268 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 904/1796] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:36 2022] DELETE /identity/v3/users/47c52e0a707f4b73bd02902565a627b8 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:36.833488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8f46f-bc55-4b3a-91f2-6d8751093c22 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:36.833898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8f46f-bc55-4b3a-91f2-6d8751093c22 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:36.834297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8f46f-bc55-4b3a-91f2-6d8751093c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:36.866054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d6e8f46f-bc55-4b3a-91f2-6d8751093c22 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:36.867420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d6e8f46f-bc55-4b3a-91f2-6d8751093c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:36.971261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 893/1797] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:37.019243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:37.021651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:37.102084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d42141-87b5-4da6-b1f7-d957fe08236a admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:37.110208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54909a6f-e90c-4189-9578-35c39230fc51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:37.110898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54909a6f-e90c-4189-9578-35c39230fc51 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:37.111175 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54909a6f-e90c-4189-9578-35c39230fc51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:37.111476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54909a6f-e90c-4189-9578-35c39230fc51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:37.134786 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54909a6f-e90c-4189-9578-35c39230fc51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:37.137521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54909a6f-e90c-4189-9578-35c39230fc51 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:37.157862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 905/1798] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:41:37.478997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700e9da-8a10-47b2-ae73-4aeed207e7b8 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:37.479557 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700e9da-8a10-47b2-ae73-4aeed207e7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:37.479557 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700e9da-8a10-47b2-ae73-4aeed207e7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:37.501008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-8700e9da-8a10-47b2-ae73-4aeed207e7b8 None None] MFA Rules not processed for user `f286fcc4b509485dbb958873a2207592`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:37.502356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8700e9da-8a10-47b2-ae73-4aeed207e7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:37.553088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:37.570887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 894/1799] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:38.093895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54909a6f-e90c-4189-9578-35c39230fc51 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:38.095674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-54909a6f-e90c-4189-9578-35c39230fc51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:38.112128 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54909a6f-e90c-4189-9578-35c39230fc51 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:38.117284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:38.120191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:38.120557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:38.120922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] PATH_INFO: `/v3/projects/e46513de29f948cbbe7217f39a8c6718` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:38.122249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e46513de29f948cbbe7217f39a8c6718)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:38.124333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:38.194463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e46513de29f948cbbe7217f39a8c6718', 'request_id': 'req-7a702211-c419-470c-885f-054bd4d7dcec'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:38.195973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:38.210865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e46513de29f948cbbe7217f39a8c6718', 'request_id': 'req-7a702211-c419-470c-885f-054bd4d7dcec'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:38.233898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e46513de29f948cbbe7217f39a8c6718', 'request_id': 'req-7a702211-c419-470c-885f-054bd4d7dcec'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:38.257570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] The token cache is being invalidate because project e46513de29f948cbbe7217f39a8c6718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:38.257779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:38.258607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:38.260612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 906/1800] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:38 2022] DELETE /identity/v3/projects/e46513de29f948cbbe7217f39a8c6718 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:38.264440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700e9da-8a10-47b2-ae73-4aeed207e7b8 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:38.289949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700e9da-8a10-47b2-ae73-4aeed207e7b8 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:38.478528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:38.479675 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:38.480917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:38.485567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] PATH_INFO: `/v3/users/e0a32100669643ba9324f97e38aee206` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:38.487690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a32100669643ba9324f97e38aee206)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:38.492342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:38.601005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a32100669643ba9324f97e38aee206'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:38.618195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a32100669643ba9324f97e38aee206'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:38.620783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a32100669643ba9324f97e38aee206'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:38.667928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a32100669643ba9324f97e38aee206'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:38.682863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-07b20ab5-7023-4fb5-b596-b66ade682ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a32100669643ba9324f97e38aee206'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:38.694548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 895/1801] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:38 2022] DELETE /identity/v3/users/e0a32100669643ba9324f97e38aee206 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:39.187699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:39.189402 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:39.219839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a702211-c419-470c-885f-054bd4d7dcec admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:39.319648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:39.320441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:39.320858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:39.321398 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] PATH_INFO: `/v3/projects/170bd2cba16d494d868f7c27a97ea948` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:39.322750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=170bd2cba16d494d868f7c27a97ea948)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:39.325066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:39.418672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170bd2cba16d494d868f7c27a97ea948', 'request_id': 'req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:39.425733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170bd2cba16d494d868f7c27a97ea948', 'request_id': 'req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:39.443376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170bd2cba16d494d868f7c27a97ea948', 'request_id': 'req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:39.463684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] The token cache is being invalidate because project 170bd2cba16d494d868f7c27a97ea948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:39.463684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:39.463684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0f51f076-bbcc-4fd6-ae42-a96dd111b0a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:39.472221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 907/1802] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:39 2022] DELETE /identity/v3/projects/170bd2cba16d494d868f7c27a97ea948 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:39.473673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bfcbe-69fc-494f-b935-65274809d63d None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:39.473964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bfcbe-69fc-494f-b935-65274809d63d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:39.474421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bfcbe-69fc-494f-b935-65274809d63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:39.506149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-5b7bfcbe-69fc-494f-b935-65274809d63d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:39.508245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5b7bfcbe-69fc-494f-b935-65274809d63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:39.596820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 896/1803] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:39.606009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a682da4-26df-4a24-a981-dddc14216ce0 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:39.606299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a682da4-26df-4a24-a981-dddc14216ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:39.606581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a682da4-26df-4a24-a981-dddc14216ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:39.643979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1a682da4-26df-4a24-a981-dddc14216ce0 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:39.653263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1a682da4-26df-4a24-a981-dddc14216ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:39.673285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 908/1804] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:39.678111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bfcbe-69fc-494f-b935-65274809d63d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:39.692088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bfcbe-69fc-494f-b935-65274809d63d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:39.698229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0bf5d-eda9-45cf-9dd4-d130b70031f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:39.699051 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0bf5d-eda9-45cf-9dd4-d130b70031f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:39.699518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0bf5d-eda9-45cf-9dd4-d130b70031f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:39.700529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0bf5d-eda9-45cf-9dd4-d130b70031f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:39.701950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0bf5d-eda9-45cf-9dd4-d130b70031f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:39.702476 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0bf5d-eda9-45cf-9dd4-d130b70031f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:39.704310 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0bf5d-eda9-45cf-9dd4-d130b70031f8 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:39.717149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:39.737097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 897/1805] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:39.742780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a682da4-26df-4a24-a981-dddc14216ce0 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:39.766213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a682da4-26df-4a24-a981-dddc14216ce0 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:39.781578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d894c-c6a0-4a27-8122-506b2c58c1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:39.782213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d894c-c6a0-4a27-8122-506b2c58c1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:39.782499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d894c-c6a0-4a27-8122-506b2c58c1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:39.782916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d894c-c6a0-4a27-8122-506b2c58c1b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:39.783965 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d894c-c6a0-4a27-8122-506b2c58c1b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:39.785814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d894c-c6a0-4a27-8122-506b2c58c1b4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:39.844906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 909/1806] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:39 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:41:39.846470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0bf5d-eda9-45cf-9dd4-d130b70031f8 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:39.859091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0bf5d-eda9-45cf-9dd4-d130b70031f8 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:39.957086 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8950efc-54e6-42c4-8aa5-549e82014da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:39.958309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8950efc-54e6-42c4-8aa5-549e82014da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:39.958735 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8950efc-54e6-42c4-8aa5-549e82014da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:39.959150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8950efc-54e6-42c4-8aa5-549e82014da1 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:39.959961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8950efc-54e6-42c4-8aa5-549e82014da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:39.962811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8950efc-54e6-42c4-8aa5-549e82014da1 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:39.987955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 898/1807] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:39 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:39.992046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d894c-c6a0-4a27-8122-506b2c58c1b4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:40.002423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d894c-c6a0-4a27-8122-506b2c58c1b4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:40.007331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0370f64-e77d-49d6-8791-1c63a39e1f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:40.007942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0370f64-e77d-49d6-8791-1c63a39e1f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:40.008260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0370f64-e77d-49d6-8791-1c63a39e1f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:40.009795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0370f64-e77d-49d6-8791-1c63a39e1f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:40.010850 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0370f64-e77d-49d6-8791-1c63a39e1f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:40.013717 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0370f64-e77d-49d6-8791-1c63a39e1f93 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:40.026891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 910/1808] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:40.030904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8950efc-54e6-42c4-8aa5-549e82014da1 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:40.042440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8950efc-54e6-42c4-8aa5-549e82014da1 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:40.047301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ee6f8-c3f3-4ee0-9495-392ac0a769c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:40.048109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ee6f8-c3f3-4ee0-9495-392ac0a769c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:40.048388 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ee6f8-c3f3-4ee0-9495-392ac0a769c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:40.048662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ee6f8-c3f3-4ee0-9495-392ac0a769c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:40.049357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ee6f8-c3f3-4ee0-9495-392ac0a769c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:40.051150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ee6f8-c3f3-4ee0-9495-392ac0a769c2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:40.067180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 899/1809] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:41:40.068572 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0370f64-e77d-49d6-8791-1c63a39e1f93 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:40.095463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0370f64-e77d-49d6-8791-1c63a39e1f93 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:40.107674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f303d7-d3f4-458c-a7be-d6f06c29c847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:40.108832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f303d7-d3f4-458c-a7be-d6f06c29c847 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:40.109412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f303d7-d3f4-458c-a7be-d6f06c29c847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:40.109912 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f303d7-d3f4-458c-a7be-d6f06c29c847 admin admin] PATH_INFO: `/v3/projects/393b5dbd5f7147e4b90bf7a354f9b6fb/users/055ccf228f2a44489a1be085c457264e/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:40.134887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f303d7-d3f4-458c-a7be-d6f06c29c847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=393b5dbd5f7147e4b90bf7a354f9b6fb, user_id=055ccf228f2a44489a1be085c457264e, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:40.138920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f303d7-d3f4-458c-a7be-d6f06c29c847 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:40.163303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 911/1810] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:40 2022] PUT /identity/v3/projects/393b5dbd5f7147e4b90bf7a354f9b6fb/users/055ccf228f2a44489a1be085c457264e/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:40.175707 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fef30-40d3-4282-889e-f67c1ed60e5b None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:40.176011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fef30-40d3-4282-889e-f67c1ed60e5b None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:40.176261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fef30-40d3-4282-889e-f67c1ed60e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:40.209384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ad4fef30-40d3-4282-889e-f67c1ed60e5b None None] MFA Rules not processed for user `055ccf228f2a44489a1be085c457264e`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:40.210681 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ad4fef30-40d3-4282-889e-f67c1ed60e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:40.307116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 900/1811] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:40.318922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f303d7-d3f4-458c-a7be-d6f06c29c847 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:40.322270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e7f303d7-d3f4-458c-a7be-d6f06c29c847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:40.330628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:40.394986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f303d7-d3f4-458c-a7be-d6f06c29c847 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:40.399842 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f1efc-cb41-498c-a89d-4fee512bc329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:40.401120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f1efc-cb41-498c-a89d-4fee512bc329 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:40.401637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f1efc-cb41-498c-a89d-4fee512bc329 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:40.401941 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f1efc-cb41-498c-a89d-4fee512bc329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:40.412591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f1efc-cb41-498c-a89d-4fee512bc329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:40.414471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f1efc-cb41-498c-a89d-4fee512bc329 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:40.436571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 912/1812] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:40.451430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fef30-40d3-4282-889e-f67c1ed60e5b None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:40.539994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fef30-40d3-4282-889e-f67c1ed60e5b None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:40.545260 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:40.545908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:40.546193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:40.546467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] PATH_INFO: `/v3/users/055ccf228f2a44489a1be085c457264e` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:40.547888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=055ccf228f2a44489a1be085c457264e)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:40.549888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:40.555732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:40.608704 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055ccf228f2a44489a1be085c457264e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:40.617492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055ccf228f2a44489a1be085c457264e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:40.618215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055ccf228f2a44489a1be085c457264e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:40.632434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055ccf228f2a44489a1be085c457264e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:40.640383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055ccf228f2a44489a1be085c457264e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:40.658809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 901/1813] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:40 2022] DELETE /identity/v3/users/055ccf228f2a44489a1be085c457264e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:40.664490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08439d-f855-4890-8ea2-08959128bf08 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:40.664810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08439d-f855-4890-8ea2-08959128bf08 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:40.665099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08439d-f855-4890-8ea2-08959128bf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:40.689948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-2c08439d-f855-4890-8ea2-08959128bf08 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:40.691391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2c08439d-f855-4890-8ea2-08959128bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:40.763294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 913/1814] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:40.777736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:40.779267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:40.861519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143905db-3d6c-401f-81ae-0b85c56092b8 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:40.865822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e8e92-dc92-4c0d-b0e6-1aa1d4ca76de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:40.866532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e8e92-dc92-4c0d-b0e6-1aa1d4ca76de service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:40.866877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e8e92-dc92-4c0d-b0e6-1aa1d4ca76de service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:40.867221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e8e92-dc92-4c0d-b0e6-1aa1d4ca76de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:40.883917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e8e92-dc92-4c0d-b0e6-1aa1d4ca76de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:40.886498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e8e92-dc92-4c0d-b0e6-1aa1d4ca76de service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:40.905102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 902/1815] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:41.449331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08439d-f855-4890-8ea2-08959128bf08 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:41.451369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2c08439d-f855-4890-8ea2-08959128bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:41.471928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08439d-f855-4890-8ea2-08959128bf08 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:41.481078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:41.481746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:41.482021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:41.482613 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] PATH_INFO: `/v3/projects/393b5dbd5f7147e4b90bf7a354f9b6fb` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:41.484039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] RBAC: Authorizing `identity:delete_project(project_id=393b5dbd5f7147e4b90bf7a354f9b6fb)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:41.486290 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:41.554818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '393b5dbd5f7147e4b90bf7a354f9b6fb', 'request_id': 'req-7186f2ca-8c8e-4337-937a-37cf1d01349f'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:41.566543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '393b5dbd5f7147e4b90bf7a354f9b6fb', 'request_id': 'req-7186f2ca-8c8e-4337-937a-37cf1d01349f'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:41.576332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '393b5dbd5f7147e4b90bf7a354f9b6fb', 'request_id': 'req-7186f2ca-8c8e-4337-937a-37cf1d01349f'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:41.590797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] The token cache is being invalidate because project 393b5dbd5f7147e4b90bf7a354f9b6fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:41.591045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:41.591894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7186f2ca-8c8e-4337-937a-37cf1d01349f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:41.596958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 914/1816] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:41 2022] DELETE /identity/v3/projects/393b5dbd5f7147e4b90bf7a354f9b6fb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:41.598498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927d612-7775-4c6c-b4dc-6a80418e1bd0 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:41.598944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927d612-7775-4c6c-b4dc-6a80418e1bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:41.599022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927d612-7775-4c6c-b4dc-6a80418e1bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:41.632506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d927d612-7775-4c6c-b4dc-6a80418e1bd0 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:41.633797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d927d612-7775-4c6c-b4dc-6a80418e1bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:41.747822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 903/1817] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:41.755466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3904428-4663-462e-b637-f3a217c1a665 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:41.755747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3904428-4663-462e-b637-f3a217c1a665 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:41.756022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3904428-4663-462e-b637-f3a217c1a665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:41.795917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d3904428-4663-462e-b637-f3a217c1a665 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:41.798429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d3904428-4663-462e-b637-f3a217c1a665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:41.818323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 915/1818] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:41.823407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927d612-7775-4c6c-b4dc-6a80418e1bd0 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:41.836245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927d612-7775-4c6c-b4dc-6a80418e1bd0 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:41.841641 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5f7f2-809b-4dbf-9036-36907da9272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:41.853966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5f7f2-809b-4dbf-9036-36907da9272e admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:41.854298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5f7f2-809b-4dbf-9036-36907da9272e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:41.856537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5f7f2-809b-4dbf-9036-36907da9272e admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:41.857423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5f7f2-809b-4dbf-9036-36907da9272e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:41.857738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5f7f2-809b-4dbf-9036-36907da9272e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:41.877783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5f7f2-809b-4dbf-9036-36907da9272e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:41.894202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 904/1819] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:41.898699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3904428-4663-462e-b637-f3a217c1a665 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:41.909233 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3904428-4663-462e-b637-f3a217c1a665 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:41.936212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f80f8-8581-4693-b63e-5a72e32307a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:41.937170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f80f8-8581-4693-b63e-5a72e32307a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:41.937755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f80f8-8581-4693-b63e-5a72e32307a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:41.937755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f80f8-8581-4693-b63e-5a72e32307a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:41.938517 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f80f8-8581-4693-b63e-5a72e32307a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:41.940617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f80f8-8581-4693-b63e-5a72e32307a9 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:41.994320 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 916/1820] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:41 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:41.998374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5f7f2-809b-4dbf-9036-36907da9272e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:42.008534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5f7f2-809b-4dbf-9036-36907da9272e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:42.022897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:42.075123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f80f8-8581-4693-b63e-5a72e32307a9 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:42.077408 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c64f80f8-8581-4693-b63e-5a72e32307a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:42.082729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:42.123304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c4915-cbc5-4301-8f7a-583140f62e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:42.125953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c4915-cbc5-4301-8f7a-583140f62e20 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:42.125953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c4915-cbc5-4301-8f7a-583140f62e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:42.125953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c4915-cbc5-4301-8f7a-583140f62e20 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:42.125953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c4915-cbc5-4301-8f7a-583140f62e20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:42.127207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c4915-cbc5-4301-8f7a-583140f62e20 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:42.175140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 905/1821] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:41 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:42.179053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c4915-cbc5-4301-8f7a-583140f62e20 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:42.198882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f80f8-8581-4693-b63e-5a72e32307a9 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:42.204417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c4915-cbc5-4301-8f7a-583140f62e20 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:42.209014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:42.209807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:42.210226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:42.210569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] PATH_INFO: `/v3/users/47bc074aee7d427bb0336fd19d31ef9d` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:42.211823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47bc074aee7d427bb0336fd19d31ef9d)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:42.216002 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:42.224163 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b9568-1d2b-4a6c-b185-a271d7a413c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:42.224825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b9568-1d2b-4a6c-b185-a271d7a413c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:42.248677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b9568-1d2b-4a6c-b185-a271d7a413c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:42.251330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b9568-1d2b-4a6c-b185-a271d7a413c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:42.252072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b9568-1d2b-4a6c-b185-a271d7a413c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:42.254076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b9568-1d2b-4a6c-b185-a271d7a413c4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:42.287728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 906/1822] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:42.288688 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b9568-1d2b-4a6c-b185-a271d7a413c4 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:42.304527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47bc074aee7d427bb0336fd19d31ef9d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:42.308947 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b9568-1d2b-4a6c-b185-a271d7a413c4 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:42.316273 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54a5b3-9fac-424d-a75f-0f64966d5ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:42.318021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54a5b3-9fac-424d-a75f-0f64966d5ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:42.318539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54a5b3-9fac-424d-a75f-0f64966d5ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:42.319182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54a5b3-9fac-424d-a75f-0f64966d5ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:42.320505 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54a5b3-9fac-424d-a75f-0f64966d5ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:42.325527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54a5b3-9fac-424d-a75f-0f64966d5ac7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:42.325727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47bc074aee7d427bb0336fd19d31ef9d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:42.327341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47bc074aee7d427bb0336fd19d31ef9d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:42.344885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 907/1823] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:42.346019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47bc074aee7d427bb0336fd19d31ef9d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:42.346828 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0c30403e-7677-4ab7-8f8b-595df87f02c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47bc074aee7d427bb0336fd19d31ef9d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:42.349752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54a5b3-9fac-424d-a75f-0f64966d5ac7 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:42.353594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2e54a5b3-9fac-424d-a75f-0f64966d5ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:42.374045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 917/1824] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:42 2022] DELETE /identity/v3/users/47bc074aee7d427bb0336fd19d31ef9d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:42.389692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54a5b3-9fac-424d-a75f-0f64966d5ac7 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:42.469290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca62bf-2771-4811-b88b-a9949cf10742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:42.470234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca62bf-2771-4811-b88b-a9949cf10742 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:42.470667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca62bf-2771-4811-b88b-a9949cf10742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:42.471869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca62bf-2771-4811-b88b-a9949cf10742 admin admin] PATH_INFO: `/v3/projects/cb5dc13f66314c6fbeb8b229a18a973d/users/e39486ccc74e44cf95a3b851481e76f0/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:42.508595 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca62bf-2771-4811-b88b-a9949cf10742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5dc13f66314c6fbeb8b229a18a973d, user_id=e39486ccc74e44cf95a3b851481e76f0, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:42.511574 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca62bf-2771-4811-b88b-a9949cf10742 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:42.533468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 908/1825] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:42 2022] PUT /identity/v3/projects/cb5dc13f66314c6fbeb8b229a18a973d/users/e39486ccc74e44cf95a3b851481e76f0/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:42.544381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988e739-3f85-442e-b7b9-221feba8ec5b None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:42.545091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988e739-3f85-442e-b7b9-221feba8ec5b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:42.545091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988e739-3f85-442e-b7b9-221feba8ec5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:42.606118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-2988e739-3f85-442e-b7b9-221feba8ec5b None None] MFA Rules not processed for user `e39486ccc74e44cf95a3b851481e76f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:42.607520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2988e739-3f85-442e-b7b9-221feba8ec5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:42.689770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 918/1826] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:42.701747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca62bf-2771-4811-b88b-a9949cf10742 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:42.703534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-82ca62bf-2771-4811-b88b-a9949cf10742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:42.765664 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca62bf-2771-4811-b88b-a9949cf10742 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:42.769398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1c04c-1ca4-44ad-87b6-e694660f8c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:42.770014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1c04c-1ca4-44ad-87b6-e694660f8c35 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:42.770290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1c04c-1ca4-44ad-87b6-e694660f8c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:42.770557 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1c04c-1ca4-44ad-87b6-e694660f8c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:42.780218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1c04c-1ca4-44ad-87b6-e694660f8c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:42.781700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1c04c-1ca4-44ad-87b6-e694660f8c35 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:42.792219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 909/1827] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:42.818414 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988e739-3f85-442e-b7b9-221feba8ec5b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:42.831449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:42.905694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988e739-3f85-442e-b7b9-221feba8ec5b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:42.910728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:42.911367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:42.911648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:42.911920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] PATH_INFO: `/v3/users/e39486ccc74e44cf95a3b851481e76f0` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:42.913093 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39486ccc74e44cf95a3b851481e76f0)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:42.915007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:42.979457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39486ccc74e44cf95a3b851481e76f0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:42.999002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1c04c-1ca4-44ad-87b6-e694660f8c35 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:43.003293 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d5f1c04c-1ca4-44ad-87b6-e694660f8c35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:43.003481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39486ccc74e44cf95a3b851481e76f0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:43.004178 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39486ccc74e44cf95a3b851481e76f0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:43.014050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39486ccc74e44cf95a3b851481e76f0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:43.014788 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0b15f582-f7d4-4483-b5ec-e8495b920272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39486ccc74e44cf95a3b851481e76f0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:43.038615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1c04c-1ca4-44ad-87b6-e694660f8c35 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:43.044300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:43.044975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.045257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.045530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] PATH_INFO: `/v3/projects/6c580418dbc24525b6631dcd0a5f7876` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.046699 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c580418dbc24525b6631dcd0a5f7876)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:43.048617 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:43.056001 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:43.059583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 919/1828] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:42 2022] DELETE /identity/v3/users/e39486ccc74e44cf95a3b851481e76f0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:43.064990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca21f8c-4cc0-43c2-bb09-467f9cc836c4 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.065260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca21f8c-4cc0-43c2-bb09-467f9cc836c4 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.065586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca21f8c-4cc0-43c2-bb09-467f9cc836c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.100768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-dca21f8c-4cc0-43c2-bb09-467f9cc836c4 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:43.102466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-dca21f8c-4cc0-43c2-bb09-467f9cc836c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:43.125792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c580418dbc24525b6631dcd0a5f7876', 'request_id': 'req-d434cb34-8a5a-457c-9e65-6914ca144226'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:43.139256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c580418dbc24525b6631dcd0a5f7876', 'request_id': 'req-d434cb34-8a5a-457c-9e65-6914ca144226'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:43.152514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c580418dbc24525b6631dcd0a5f7876', 'request_id': 'req-d434cb34-8a5a-457c-9e65-6914ca144226'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:43.174120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] The token cache is being invalidate because project 6c580418dbc24525b6631dcd0a5f7876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:43.180637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:43.183324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d434cb34-8a5a-457c-9e65-6914ca144226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:43.185428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 910/1829] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:42 2022] DELETE /identity/v3/projects/6c580418dbc24525b6631dcd0a5f7876 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:43.190430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e58de-6d38-4da9-8d57-74b6fad9be37 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.190727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e58de-6d38-4da9-8d57-74b6fad9be37 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.191164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e58de-6d38-4da9-8d57-74b6fad9be37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.205324 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 920/1830] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:43.218515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca21f8c-4cc0-43c2-bb09-467f9cc836c4 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:43.220238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-dca21f8c-4cc0-43c2-bb09-467f9cc836c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:43.223934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a97e58de-6d38-4da9-8d57-74b6fad9be37 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:43.225901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a97e58de-6d38-4da9-8d57-74b6fad9be37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:43.266748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:43.301241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 911/1831] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:43.310781 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdbe45-7774-4543-b0fb-2468c82cc7da None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.311523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca21f8c-4cc0-43c2-bb09-467f9cc836c4 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:43.312561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdbe45-7774-4543-b0fb-2468c82cc7da None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.313114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdbe45-7774-4543-b0fb-2468c82cc7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.342876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-c1bdbe45-7774-4543-b0fb-2468c82cc7da None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:43.345007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c1bdbe45-7774-4543-b0fb-2468c82cc7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:43.357743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 912/1832] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:43.362930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdbe45-7774-4543-b0fb-2468c82cc7da None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:43.374215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdbe45-7774-4543-b0fb-2468c82cc7da None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:43.379326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c63ba-9b2b-4770-bef1-2fafe8811564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:43.379944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c63ba-9b2b-4770-bef1-2fafe8811564 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.380217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c63ba-9b2b-4770-bef1-2fafe8811564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.380601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c63ba-9b2b-4770-bef1-2fafe8811564 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.381395 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c63ba-9b2b-4770-bef1-2fafe8811564 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:43.381740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072ecda-fd63-42a4-8078-12a00b1fdfb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:43.381866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c63ba-9b2b-4770-bef1-2fafe8811564 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:43.383377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072ecda-fd63-42a4-8078-12a00b1fdfb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.383906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c63ba-9b2b-4770-bef1-2fafe8811564 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:43.384170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072ecda-fd63-42a4-8078-12a00b1fdfb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.384627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072ecda-fd63-42a4-8078-12a00b1fdfb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.396685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072ecda-fd63-42a4-8078-12a00b1fdfb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:43.403630 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072ecda-fd63-42a4-8078-12a00b1fdfb2 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:43.410073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 913/1833] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:43.413659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 921/1834] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:43.417069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c63ba-9b2b-4770-bef1-2fafe8811564 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:43.427388 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c63ba-9b2b-4770-bef1-2fafe8811564 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:43.434702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c2e0-7381-48c0-a870-881fe79977ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:43.435322 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c2e0-7381-48c0-a870-881fe79977ac admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.436025 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c2e0-7381-48c0-a870-881fe79977ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.436305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c2e0-7381-48c0-a870-881fe79977ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.436961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c2e0-7381-48c0-a870-881fe79977ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:43.441355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c2e0-7381-48c0-a870-881fe79977ac admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:43.483731 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 914/1835] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:43 2022] POST /identity/v3/projects => generated 389 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:43.489438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072ecda-fd63-42a4-8078-12a00b1fdfb2 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:43.512601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072ecda-fd63-42a4-8078-12a00b1fdfb2 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:43.625216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ae7ec-e6dd-4eb6-8549-d98adfefcf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:43.625870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ae7ec-e6dd-4eb6-8549-d98adfefcf5d admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.626124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ae7ec-e6dd-4eb6-8549-d98adfefcf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.626368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ae7ec-e6dd-4eb6-8549-d98adfefcf5d admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.627412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ae7ec-e6dd-4eb6-8549-d98adfefcf5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:43.629245 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ae7ec-e6dd-4eb6-8549-d98adfefcf5d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:43.660432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 922/1836] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:43 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:41:43.664600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c2e0-7381-48c0-a870-881fe79977ac admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:43.687142 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c2e0-7381-48c0-a870-881fe79977ac admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:43.698166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e9240-0e18-4c51-a8f0-ca2b8ffd11c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:43.698797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e9240-0e18-4c51-a8f0-ca2b8ffd11c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.699074 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e9240-0e18-4c51-a8f0-ca2b8ffd11c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.699341 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e9240-0e18-4c51-a8f0-ca2b8ffd11c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.704965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e9240-0e18-4c51-a8f0-ca2b8ffd11c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:43.704965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e9240-0e18-4c51-a8f0-ca2b8ffd11c4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:43.725608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 915/1837] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:41:43.728516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ae7ec-e6dd-4eb6-8549-d98adfefcf5d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:43.739327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ae7ec-e6dd-4eb6-8549-d98adfefcf5d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:43.748220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec909a1-a632-42e5-b2c5-5e471ef44ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:43.749121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec909a1-a632-42e5-b2c5-5e471ef44ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.749519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec909a1-a632-42e5-b2c5-5e471ef44ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.749914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec909a1-a632-42e5-b2c5-5e471ef44ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.750666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec909a1-a632-42e5-b2c5-5e471ef44ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:43.752385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec909a1-a632-42e5-b2c5-5e471ef44ce1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:43.768904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 923/1838] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:43.772365 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e9240-0e18-4c51-a8f0-ca2b8ffd11c4 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:43.784185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e9240-0e18-4c51-a8f0-ca2b8ffd11c4 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:43.789310 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103252ff-dad0-4554-b313-9a1493318994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:43.790161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103252ff-dad0-4554-b313-9a1493318994 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.790554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103252ff-dad0-4554-b313-9a1493318994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.790910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103252ff-dad0-4554-b313-9a1493318994 admin admin] PATH_INFO: `/v3/projects/c84cfe68e5db4b2dba60443b48d23097/users/72d748ebce2742058757304095d1a641/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.812375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103252ff-dad0-4554-b313-9a1493318994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84cfe68e5db4b2dba60443b48d23097, user_id=72d748ebce2742058757304095d1a641, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:43.815702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103252ff-dad0-4554-b313-9a1493318994 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:43.839274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 916/1839] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:43 2022] PUT /identity/v3/projects/c84cfe68e5db4b2dba60443b48d23097/users/72d748ebce2742058757304095d1a641/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:43.842514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e106da6-df16-43ab-bd51-39df1ac6770b None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:43.843977 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e106da6-df16-43ab-bd51-39df1ac6770b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:43.844428 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e106da6-df16-43ab-bd51-39df1ac6770b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:43.869069 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6e106da6-df16-43ab-bd51-39df1ac6770b None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:43.870915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6e106da6-df16-43ab-bd51-39df1ac6770b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:43.970116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103252ff-dad0-4554-b313-9a1493318994 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:43.972118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-103252ff-dad0-4554-b313-9a1493318994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:43.976728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 924/1840] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:43.989544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e106da6-df16-43ab-bd51-39df1ac6770b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:43.991378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6e106da6-df16-43ab-bd51-39df1ac6770b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:44.013969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103252ff-dad0-4554-b313-9a1493318994 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:44.015239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:44.019541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:44.020273 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.020712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.021065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] PATH_INFO: `/v3/projects/cb5dc13f66314c6fbeb8b229a18a973d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.022413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5dc13f66314c6fbeb8b229a18a973d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:44.024465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:44.036204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:44.080962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e106da6-df16-43ab-bd51-39df1ac6770b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:44.094542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5dc13f66314c6fbeb8b229a18a973d', 'request_id': 'req-c2ea9b60-3118-41cb-b977-9e0b93419eda'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:44.105270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5dc13f66314c6fbeb8b229a18a973d', 'request_id': 'req-c2ea9b60-3118-41cb-b977-9e0b93419eda'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:44.117094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5dc13f66314c6fbeb8b229a18a973d', 'request_id': 'req-c2ea9b60-3118-41cb-b977-9e0b93419eda'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:44.138033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] The token cache is being invalidate because project cb5dc13f66314c6fbeb8b229a18a973d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:44.138326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:44.139120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2ea9b60-3118-41cb-b977-9e0b93419eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:44.141140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 917/1841] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:43 2022] DELETE /identity/v3/projects/cb5dc13f66314c6fbeb8b229a18a973d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:44.145703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a17aa0-3e3b-4634-a554-965d1dbf5887 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.146052 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a17aa0-3e3b-4634-a554-965d1dbf5887 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.146385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a17aa0-3e3b-4634-a554-965d1dbf5887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.150530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2c272-225a-44ee-83e3-4eb7b02b582f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:44.151141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2c272-225a-44ee-83e3-4eb7b02b582f service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.151417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2c272-225a-44ee-83e3-4eb7b02b582f service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.151721 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2c272-225a-44ee-83e3-4eb7b02b582f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.153474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7fe2c272-225a-44ee-83e3-4eb7b02b582f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:44.184233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-56a17aa0-3e3b-4634-a554-965d1dbf5887 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:44.185513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-56a17aa0-3e3b-4634-a554-965d1dbf5887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:44.256616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2c272-225a-44ee-83e3-4eb7b02b582f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:44.258625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2c272-225a-44ee-83e3-4eb7b02b582f service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:44.268721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 918/1842] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:44.274797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 925/1843] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:44.277738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afef9b7-0224-4753-8686-d8bd61df21f8 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.278272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afef9b7-0224-4753-8686-d8bd61df21f8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.281199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afef9b7-0224-4753-8686-d8bd61df21f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.303018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-6afef9b7-0224-4753-8686-d8bd61df21f8 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:44.304753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6afef9b7-0224-4753-8686-d8bd61df21f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:44.315864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 919/1844] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:44.321263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2c272-225a-44ee-83e3-4eb7b02b582f service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:44.332590 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2c272-225a-44ee-83e3-4eb7b02b582f service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:44.338848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e301a90-08c4-4242-bd67-aa68f9bfb3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:44.339891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e301a90-08c4-4242-bd67-aa68f9bfb3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.340294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e301a90-08c4-4242-bd67-aa68f9bfb3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.340665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e301a90-08c4-4242-bd67-aa68f9bfb3f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.341499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e301a90-08c4-4242-bd67-aa68f9bfb3f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:44.341910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e301a90-08c4-4242-bd67-aa68f9bfb3f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:44.343966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e301a90-08c4-4242-bd67-aa68f9bfb3f2 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:44.362610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 926/1845] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:44.367713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afef9b7-0224-4753-8686-d8bd61df21f8 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:44.381058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afef9b7-0224-4753-8686-d8bd61df21f8 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:44.392420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f72d4-ac1a-4054-a8f9-bb47ae0b55d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:44.393761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f72d4-ac1a-4054-a8f9-bb47ae0b55d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.393761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f72d4-ac1a-4054-a8f9-bb47ae0b55d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.393761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f72d4-ac1a-4054-a8f9-bb47ae0b55d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.395080 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f72d4-ac1a-4054-a8f9-bb47ae0b55d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:44.397291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f72d4-ac1a-4054-a8f9-bb47ae0b55d1 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:44.449079 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 920/1846] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:44 2022] POST /identity/v3/projects => generated 375 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:44.453077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e301a90-08c4-4242-bd67-aa68f9bfb3f2 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:44.467578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e301a90-08c4-4242-bd67-aa68f9bfb3f2 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:44.529001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa9a02-357b-474b-a021-edecebfeff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:44.529615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa9a02-357b-474b-a021-edecebfeff51 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.529934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa9a02-357b-474b-a021-edecebfeff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.530163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa9a02-357b-474b-a021-edecebfeff51 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.530798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa9a02-357b-474b-a021-edecebfeff51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:44.532700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa9a02-357b-474b-a021-edecebfeff51 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:44.542405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:44.562446 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 927/1847] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:44 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:44.565378 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f72d4-ac1a-4054-a8f9-bb47ae0b55d1 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:44.579189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f72d4-ac1a-4054-a8f9-bb47ae0b55d1 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:44.584074 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dada57-33a2-4898-b466-0b2bf6e415f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:44.584790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dada57-33a2-4898-b466-0b2bf6e415f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.585161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dada57-33a2-4898-b466-0b2bf6e415f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.585509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dada57-33a2-4898-b466-0b2bf6e415f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.586399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dada57-33a2-4898-b466-0b2bf6e415f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:44.588484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dada57-33a2-4898-b466-0b2bf6e415f0 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:44.600335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 921/1848] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:44.605684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa9a02-357b-474b-a021-edecebfeff51 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:44.617147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa9a02-357b-474b-a021-edecebfeff51 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:44.622562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb7a18-f1af-4818-a51a-d858cdc7dafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:44.623179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb7a18-f1af-4818-a51a-d858cdc7dafe admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.623464 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb7a18-f1af-4818-a51a-d858cdc7dafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.624000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb7a18-f1af-4818-a51a-d858cdc7dafe admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.624828 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb7a18-f1af-4818-a51a-d858cdc7dafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:44.626869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb7a18-f1af-4818-a51a-d858cdc7dafe admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:44.649602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 928/1849] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:44.654067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dada57-33a2-4898-b466-0b2bf6e415f0 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:44.670444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dada57-33a2-4898-b466-0b2bf6e415f0 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:44.675839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca44ab4-3285-482d-ad80-4c2db98170e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:44.676625 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca44ab4-3285-482d-ad80-4c2db98170e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.677164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca44ab4-3285-482d-ad80-4c2db98170e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.677478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca44ab4-3285-482d-ad80-4c2db98170e2 admin admin] PATH_INFO: `/v3/projects/609b9817eac74a128f0c25ebaff4b91d/users/852d0ec5ac6d44f4bcc01ab3a3e1fe32/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.693622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca44ab4-3285-482d-ad80-4c2db98170e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609b9817eac74a128f0c25ebaff4b91d, user_id=852d0ec5ac6d44f4bcc01ab3a3e1fe32, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:44.696289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca44ab4-3285-482d-ad80-4c2db98170e2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:44.710637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 922/1850] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:44 2022] PUT /identity/v3/projects/609b9817eac74a128f0c25ebaff4b91d/users/852d0ec5ac6d44f4bcc01ab3a3e1fe32/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:44.721360 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614efdd-0964-4e5e-ad42-3c5bd42ebc0c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.721360 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614efdd-0964-4e5e-ad42-3c5bd42ebc0c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.721629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614efdd-0964-4e5e-ad42-3c5bd42ebc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.752095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-3614efdd-0964-4e5e-ad42-3c5bd42ebc0c None None] MFA Rules not processed for user `852d0ec5ac6d44f4bcc01ab3a3e1fe32`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:44.753344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3614efdd-0964-4e5e-ad42-3c5bd42ebc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:44.826043 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 929/1851] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:44.838172 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca44ab4-3285-482d-ad80-4c2db98170e2 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:44.840354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-bca44ab4-3285-482d-ad80-4c2db98170e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:44.896239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:44.919138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca44ab4-3285-482d-ad80-4c2db98170e2 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:44.925413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00688652-ed44-4684-a730-7d02f509dd0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:44.926270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00688652-ed44-4684-a730-7d02f509dd0f service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:44.927071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00688652-ed44-4684-a730-7d02f509dd0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:44.927414 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00688652-ed44-4684-a730-7d02f509dd0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:44.936981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00688652-ed44-4684-a730-7d02f509dd0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:44.938509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00688652-ed44-4684-a730-7d02f509dd0f service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:44.953765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 923/1852] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:48.370463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614efdd-0964-4e5e-ad42-3c5bd42ebc0c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:48.454740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614efdd-0964-4e5e-ad42-3c5bd42ebc0c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:48.459475 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:48.460209 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:48.460549 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:48.460932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] PATH_INFO: `/v3/users/852d0ec5ac6d44f4bcc01ab3a3e1fe32` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:48.462696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=852d0ec5ac6d44f4bcc01ab3a3e1fe32)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:48.465012 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:48.523714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '852d0ec5ac6d44f4bcc01ab3a3e1fe32'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:48.540101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '852d0ec5ac6d44f4bcc01ab3a3e1fe32'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:48.541039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '852d0ec5ac6d44f4bcc01ab3a3e1fe32'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:48.546734 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:48.549022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '852d0ec5ac6d44f4bcc01ab3a3e1fe32'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:48.550244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '852d0ec5ac6d44f4bcc01ab3a3e1fe32'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:48.570251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 930/1853] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:48 2022] DELETE /identity/v3/users/852d0ec5ac6d44f4bcc01ab3a3e1fe32 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:48.575962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18906f9a-c57e-40a6-a7cc-a3518d5945d7 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:48.577050 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18906f9a-c57e-40a6-a7cc-a3518d5945d7 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:48.583046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18906f9a-c57e-40a6-a7cc-a3518d5945d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:48.607043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-18906f9a-c57e-40a6-a7cc-a3518d5945d7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:48.609406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-18906f9a-c57e-40a6-a7cc-a3518d5945d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:48.732084 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 924/1854] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:48.753126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:48.756993 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:48.865374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18384cfc-cd16-4ff4-9d3b-0dc55a9fb1fb admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:48.892614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fea7da-ce1c-4b67-8958-3b79fb499c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:48.893426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fea7da-ce1c-4b67-8958-3b79fb499c08 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:48.893781 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fea7da-ce1c-4b67-8958-3b79fb499c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:48.894103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fea7da-ce1c-4b67-8958-3b79fb499c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:48.912434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fea7da-ce1c-4b67-8958-3b79fb499c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:48.914648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fea7da-ce1c-4b67-8958-3b79fb499c08 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:48.934691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 931/1855] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:49.785217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18906f9a-c57e-40a6-a7cc-a3518d5945d7 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:49.788179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-18906f9a-c57e-40a6-a7cc-a3518d5945d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:49.809040 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18906f9a-c57e-40a6-a7cc-a3518d5945d7 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:49.815900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:49.817417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:49.817929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:49.819160 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] PATH_INFO: `/v3/projects/609b9817eac74a128f0c25ebaff4b91d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:49.822819 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] RBAC: Authorizing `identity:delete_project(project_id=609b9817eac74a128f0c25ebaff4b91d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:49.826916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:49.944869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609b9817eac74a128f0c25ebaff4b91d', 'request_id': 'req-f39c4c9b-fd19-4e00-8b51-2913692ac12e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:49.954316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609b9817eac74a128f0c25ebaff4b91d', 'request_id': 'req-f39c4c9b-fd19-4e00-8b51-2913692ac12e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:49.970398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609b9817eac74a128f0c25ebaff4b91d', 'request_id': 'req-f39c4c9b-fd19-4e00-8b51-2913692ac12e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:50.009805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] The token cache is being invalidate because project 609b9817eac74a128f0c25ebaff4b91d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:50.012136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:50.013321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:50.017488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 925/1856] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:49 2022] DELETE /identity/v3/projects/609b9817eac74a128f0c25ebaff4b91d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:50.688654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ef6c1-25f3-4fb6-a5dc-491f8e004bc5 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:50.689158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ef6c1-25f3-4fb6-a5dc-491f8e004bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:50.689429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ef6c1-25f3-4fb6-a5dc-491f8e004bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:50.723233 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-394ef6c1-25f3-4fb6-a5dc-491f8e004bc5 None None] MFA Rules not processed for user `72d748ebce2742058757304095d1a641`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:50.724720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-394ef6c1-25f3-4fb6-a5dc-491f8e004bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:50.850757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 932/1857] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:50.866178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:50.867633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:50.990202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:51.028087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c4c9b-fd19-4e00-8b51-2913692ac12e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:51.034758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a5fec-b152-4f87-bfda-1afc01210591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:51.035404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a5fec-b152-4f87-bfda-1afc01210591 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:51.035682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a5fec-b152-4f87-bfda-1afc01210591 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:51.035952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a5fec-b152-4f87-bfda-1afc01210591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:51.045463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a5fec-b152-4f87-bfda-1afc01210591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:51.047102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a5fec-b152-4f87-bfda-1afc01210591 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:51.057785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 926/1858] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:52.581309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf1c0-f707-4193-8a9b-9ab3329f067f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:52.582306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf1c0-f707-4193-8a9b-9ab3329f067f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:52.582851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf1c0-f707-4193-8a9b-9ab3329f067f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:52.635288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d9bcf1c0-f707-4193-8a9b-9ab3329f067f None None] MFA Rules not processed for user `72d748ebce2742058757304095d1a641`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:52.636985 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d9bcf1c0-f707-4193-8a9b-9ab3329f067f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:52.645232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 933/1859] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:54.661622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a5fec-b152-4f87-bfda-1afc01210591 service nova] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:54.663042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ac4a5fec-b152-4f87-bfda-1afc01210591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:54.740340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f9e97-f887-4baa-b5c4-711902ae418e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:54.740985 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f9e97-f887-4baa-b5c4-711902ae418e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:54.740985 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f9e97-f887-4baa-b5c4-711902ae418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:54.744032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a5fec-b152-4f87-bfda-1afc01210591 service nova] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:54.749106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:54.749752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:54.750029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:54.750300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] PATH_INFO: `/v3/users/a196f0cc47c54c11bd618f7f33f8bdb2` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:54.751458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a196f0cc47c54c11bd618f7f33f8bdb2)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:54.753488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:54.762507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-a85f9e97-f887-4baa-b5c4-711902ae418e None None] MFA Rules not processed for user `fcb7dccab5ec46348c87ad4889120109`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:54.764090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a85f9e97-f887-4baa-b5c4-711902ae418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:54.831098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 934/1860] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:54.833629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a196f0cc47c54c11bd618f7f33f8bdb2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:54.844988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a196f0cc47c54c11bd618f7f33f8bdb2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:54.846109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a196f0cc47c54c11bd618f7f33f8bdb2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:54.864457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a196f0cc47c54c11bd618f7f33f8bdb2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:54.884703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a196f0cc47c54c11bd618f7f33f8bdb2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:54.902217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 927/1861] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:41:54 2022] DELETE /identity/v3/users/a196f0cc47c54c11bd618f7f33f8bdb2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:54.907401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d0268-4f37-4e39-9bea-312254359c21 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:54.907848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d0268-4f37-4e39-9bea-312254359c21 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:54.908443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d0268-4f37-4e39-9bea-312254359c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:54.931498 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-018d0268-4f37-4e39-9bea-312254359c21 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:54.932835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-018d0268-4f37-4e39-9bea-312254359c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:54.947034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:55.007691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 935/1862] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:55.015025 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:55.016530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:55.082006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3d0c3-c8f3-478c-835d-a1ed9e9056ee admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:55.089125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f47d3c-30da-4518-950e-d73bb60259cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:55.090031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47d3c-30da-4518-950e-d73bb60259cc service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:55.090313 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47d3c-30da-4518-950e-d73bb60259cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:55.090589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f47d3c-30da-4518-950e-d73bb60259cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:55.100683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f47d3c-30da-4518-950e-d73bb60259cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:55.102202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f47d3c-30da-4518-950e-d73bb60259cc service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:55.112512 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 928/1863] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:55.895991 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d0268-4f37-4e39-9bea-312254359c21 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:55.897618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-018d0268-4f37-4e39-9bea-312254359c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:55.927304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d0268-4f37-4e39-9bea-312254359c21 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:55.938537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:55.939460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:55.940308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:55.941141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] PATH_INFO: `/v3/projects/5ebf17b7731d40ae9f63af508cd3bc38` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:55.944326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ebf17b7731d40ae9f63af508cd3bc38)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:55.958572 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:56.026516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ebf17b7731d40ae9f63af508cd3bc38', 'request_id': 'req-3713cd53-acf8-4183-9b91-e10ca6797336'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:56.033354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ebf17b7731d40ae9f63af508cd3bc38', 'request_id': 'req-3713cd53-acf8-4183-9b91-e10ca6797336'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:56.044542 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ebf17b7731d40ae9f63af508cd3bc38', 'request_id': 'req-3713cd53-acf8-4183-9b91-e10ca6797336'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:56.059463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] The token cache is being invalidate because project 5ebf17b7731d40ae9f63af508cd3bc38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:41:56.059838 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:56.060657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3713cd53-acf8-4183-9b91-e10ca6797336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:41:56.063110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 936/1864] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:41:55 2022] DELETE /identity/v3/projects/5ebf17b7731d40ae9f63af508cd3bc38 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:56.067848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba76a3-d895-4d15-8a13-d09d85be9d3d None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:56.068653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba76a3-d895-4d15-8a13-d09d85be9d3d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:56.069150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba76a3-d895-4d15-8a13-d09d85be9d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:56.108019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a4ba76a3-d895-4d15-8a13-d09d85be9d3d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:56.109761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a4ba76a3-d895-4d15-8a13-d09d85be9d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:56.125177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:56.236060 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 929/1865] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:56.245110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee9125-c571-410e-a028-1af5bc9dc61e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:56.245394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee9125-c571-410e-a028-1af5bc9dc61e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:56.245667 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee9125-c571-410e-a028-1af5bc9dc61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:56.272771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d9ee9125-c571-410e-a028-1af5bc9dc61e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:56.274284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d9ee9125-c571-410e-a028-1af5bc9dc61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:56.283959 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 937/1866] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:56.289290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba76a3-d895-4d15-8a13-d09d85be9d3d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:56.302244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba76a3-d895-4d15-8a13-d09d85be9d3d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:56.315848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbedd64-277a-440a-892d-f6e73e841ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:56.316680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbedd64-277a-440a-892d-f6e73e841ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:56.317078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbedd64-277a-440a-892d-f6e73e841ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:56.317471 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbedd64-277a-440a-892d-f6e73e841ed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:56.318652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbedd64-277a-440a-892d-f6e73e841ed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:41:56.319166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbedd64-277a-440a-892d-f6e73e841ed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:56.324189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbedd64-277a-440a-892d-f6e73e841ed3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:56.363428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 930/1867] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:41:56.365007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee9125-c571-410e-a028-1af5bc9dc61e None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:56.380243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee9125-c571-410e-a028-1af5bc9dc61e None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:56.389116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331824f-17fa-4ce3-ba27-c7e56876ab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:56.390877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331824f-17fa-4ce3-ba27-c7e56876ab49 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:56.393286 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331824f-17fa-4ce3-ba27-c7e56876ab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:56.393876 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331824f-17fa-4ce3-ba27-c7e56876ab49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:56.394807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331824f-17fa-4ce3-ba27-c7e56876ab49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:56.397232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331824f-17fa-4ce3-ba27-c7e56876ab49 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:56.438688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 938/1868] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:56 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:56.443496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbedd64-277a-440a-892d-f6e73e841ed3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:56.466553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbedd64-277a-440a-892d-f6e73e841ed3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:56.574581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2f009-6c14-4ed5-aee8-ee8f1356769a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:56.575207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2f009-6c14-4ed5-aee8-ee8f1356769a admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:56.575985 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2f009-6c14-4ed5-aee8-ee8f1356769a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:56.576278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2f009-6c14-4ed5-aee8-ee8f1356769a admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:56.578013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2f009-6c14-4ed5-aee8-ee8f1356769a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:56.579159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2f009-6c14-4ed5-aee8-ee8f1356769a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:56.611087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 931/1869] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:56 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:56.612108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331824f-17fa-4ce3-ba27-c7e56876ab49 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:56.623685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331824f-17fa-4ce3-ba27-c7e56876ab49 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:56.636581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eac16b-68a3-4aad-a5cc-82ba9c2acb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:56.637216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eac16b-68a3-4aad-a5cc-82ba9c2acb02 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:56.637626 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eac16b-68a3-4aad-a5cc-82ba9c2acb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:56.637751 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eac16b-68a3-4aad-a5cc-82ba9c2acb02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:56.638411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eac16b-68a3-4aad-a5cc-82ba9c2acb02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:56.640035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eac16b-68a3-4aad-a5cc-82ba9c2acb02 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:56.652544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 939/1870] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:41:56.656585 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2f009-6c14-4ed5-aee8-ee8f1356769a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:56.674175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2f009-6c14-4ed5-aee8-ee8f1356769a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:56.682765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ab560-a68d-4973-b72b-48e0584729b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:56.683415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ab560-a68d-4973-b72b-48e0584729b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:56.683692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ab560-a68d-4973-b72b-48e0584729b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:56.683964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ab560-a68d-4973-b72b-48e0584729b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:56.684620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ab560-a68d-4973-b72b-48e0584729b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:56.686649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ab560-a68d-4973-b72b-48e0584729b7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:56.703785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 932/1871] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:56.705052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eac16b-68a3-4aad-a5cc-82ba9c2acb02 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:56.719679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eac16b-68a3-4aad-a5cc-82ba9c2acb02 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:56.725616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f7ac4-1c0f-498d-b314-af92d61753e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:56.726376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f7ac4-1c0f-498d-b314-af92d61753e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:56.726708 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f7ac4-1c0f-498d-b314-af92d61753e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:56.727606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f7ac4-1c0f-498d-b314-af92d61753e6 admin admin] PATH_INFO: `/v3/projects/34ea47b80fc840458abccbec052ab5e6/users/ed6859f1acef4c43bb500d5b9c860d23/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:56.746350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f7ac4-1c0f-498d-b314-af92d61753e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ea47b80fc840458abccbec052ab5e6, user_id=ed6859f1acef4c43bb500d5b9c860d23, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:56.749050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f7ac4-1c0f-498d-b314-af92d61753e6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:56.766872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 940/1872] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:56 2022] PUT /identity/v3/projects/34ea47b80fc840458abccbec052ab5e6/users/ed6859f1acef4c43bb500d5b9c860d23/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:56.778436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6242b2-bfbb-443b-a6fb-ca76cad2064d None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:56.779441 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6242b2-bfbb-443b-a6fb-ca76cad2064d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:56.779851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6242b2-bfbb-443b-a6fb-ca76cad2064d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:56.816892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-2b6242b2-bfbb-443b-a6fb-ca76cad2064d None None] MFA Rules not processed for user `ed6859f1acef4c43bb500d5b9c860d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:56.818757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2b6242b2-bfbb-443b-a6fb-ca76cad2064d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:56.920400 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 933/1873] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:41:56.924578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f7ac4-1c0f-498d-b314-af92d61753e6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:57.004166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:57.062333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f7ac4-1c0f-498d-b314-af92d61753e6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:57.072536 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cdcde-c1a7-4a4b-90e3-f653a6bbd319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:57.073657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cdcde-c1a7-4a4b-90e3-f653a6bbd319 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:57.075142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cdcde-c1a7-4a4b-90e3-f653a6bbd319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:57.075810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cdcde-c1a7-4a4b-90e3-f653a6bbd319 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:57.076988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cdcde-c1a7-4a4b-90e3-f653a6bbd319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:57.080305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cdcde-c1a7-4a4b-90e3-f653a6bbd319 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:57.118570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 941/1874] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:56 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:57.122252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6242b2-bfbb-443b-a6fb-ca76cad2064d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:57.130877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:57.141595 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6242b2-bfbb-443b-a6fb-ca76cad2064d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:57.254322 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad598329-3a15-48a9-8ba7-9afae3ca85fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:57.255049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad598329-3a15-48a9-8ba7-9afae3ca85fa admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:57.255411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad598329-3a15-48a9-8ba7-9afae3ca85fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:57.255730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad598329-3a15-48a9-8ba7-9afae3ca85fa admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:57.256482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad598329-3a15-48a9-8ba7-9afae3ca85fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:57.259088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad598329-3a15-48a9-8ba7-9afae3ca85fa admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:57.282472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 934/1875] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:57 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:57.289845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cdcde-c1a7-4a4b-90e3-f653a6bbd319 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:57.301789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cdcde-c1a7-4a4b-90e3-f653a6bbd319 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:57.308308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126468e-21ca-4d03-80f7-473db50f92b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:57.309123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126468e-21ca-4d03-80f7-473db50f92b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:57.309406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126468e-21ca-4d03-80f7-473db50f92b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:57.309684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126468e-21ca-4d03-80f7-473db50f92b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:57.310456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6126468e-21ca-4d03-80f7-473db50f92b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:57.314315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6126468e-21ca-4d03-80f7-473db50f92b1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:57.328529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 942/1876] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:57.332357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad598329-3a15-48a9-8ba7-9afae3ca85fa admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:57.345126 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad598329-3a15-48a9-8ba7-9afae3ca85fa admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:57.351156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154daa30-f723-4a34-97d4-91891c2bed11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:57.352381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154daa30-f723-4a34-97d4-91891c2bed11 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:57.352947 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154daa30-f723-4a34-97d4-91891c2bed11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:57.353333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154daa30-f723-4a34-97d4-91891c2bed11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:57.354088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154daa30-f723-4a34-97d4-91891c2bed11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:57.356255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154daa30-f723-4a34-97d4-91891c2bed11 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:57.388157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 935/1877] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:41:57.388590 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126468e-21ca-4d03-80f7-473db50f92b1 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:57.401227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126468e-21ca-4d03-80f7-473db50f92b1 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:57.408926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43f972-4624-4477-a5aa-56aa878ef922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:57.409829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43f972-4624-4477-a5aa-56aa878ef922 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:57.410224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43f972-4624-4477-a5aa-56aa878ef922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:57.410601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43f972-4624-4477-a5aa-56aa878ef922 admin admin] PATH_INFO: `/v3/projects/035ffd1e912a4a16b3406d80434baf35/users/901ae59ca2fa47abab8a951888566360/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:57.430567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43f972-4624-4477-a5aa-56aa878ef922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035ffd1e912a4a16b3406d80434baf35, user_id=901ae59ca2fa47abab8a951888566360, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:57.433549 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43f972-4624-4477-a5aa-56aa878ef922 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:57.452899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 943/1878] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:57 2022] PUT /identity/v3/projects/035ffd1e912a4a16b3406d80434baf35/users/901ae59ca2fa47abab8a951888566360/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:41:57.463010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027069-55b9-4c75-bb73-bfc6bb6e91d6 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:57.463724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027069-55b9-4c75-bb73-bfc6bb6e91d6 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:57.464008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c027069-55b9-4c75-bb73-bfc6bb6e91d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:57.509554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3c027069-55b9-4c75-bb73-bfc6bb6e91d6 None None] MFA Rules not processed for user `901ae59ca2fa47abab8a951888566360`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:57.510966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3c027069-55b9-4c75-bb73-bfc6bb6e91d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:57.624413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 936/1879] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:57.629115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43f972-4624-4477-a5aa-56aa878ef922 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:57.712903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43f972-4624-4477-a5aa-56aa878ef922 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:57.718591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a726675-de93-49b6-b025-d6bf949be7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:57.719214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726675-de93-49b6-b025-d6bf949be7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:57.719515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726675-de93-49b6-b025-d6bf949be7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:57.719790 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726675-de93-49b6-b025-d6bf949be7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:57.720431 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726675-de93-49b6-b025-d6bf949be7c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:57.722399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726675-de93-49b6-b025-d6bf949be7c2 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:57.756028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 944/1880] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:41:57 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:57.760580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c027069-55b9-4c75-bb73-bfc6bb6e91d6 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:57.771798 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c027069-55b9-4c75-bb73-bfc6bb6e91d6 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:57.833612 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d15ed2-bda8-417e-9896-bc5edae9a608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:57.834901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d15ed2-bda8-417e-9896-bc5edae9a608 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:57.835303 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d15ed2-bda8-417e-9896-bc5edae9a608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:57.835575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d15ed2-bda8-417e-9896-bc5edae9a608 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:57.836222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d15ed2-bda8-417e-9896-bc5edae9a608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:57.838425 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d15ed2-bda8-417e-9896-bc5edae9a608 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:57.865690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 937/1881] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:41:57 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:41:57.870291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a726675-de93-49b6-b025-d6bf949be7c2 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:57.881128 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a726675-de93-49b6-b025-d6bf949be7c2 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:57.887204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def22d65-9b1c-4cc3-a630-3e819c6cac8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:57.888059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def22d65-9b1c-4cc3-a630-3e819c6cac8b admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:57.888345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def22d65-9b1c-4cc3-a630-3e819c6cac8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:57.888902 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def22d65-9b1c-4cc3-a630-3e819c6cac8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:57.889534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def22d65-9b1c-4cc3-a630-3e819c6cac8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:57.891526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def22d65-9b1c-4cc3-a630-3e819c6cac8b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:57.903183 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 945/1882] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:57.907445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d15ed2-bda8-417e-9896-bc5edae9a608 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:57.912731 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:57.920195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d15ed2-bda8-417e-9896-bc5edae9a608 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:57.925436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892c00f-a4ca-43e4-95d7-426af153b0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:57.926145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892c00f-a4ca-43e4-95d7-426af153b0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:57.926448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892c00f-a4ca-43e4-95d7-426af153b0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:57.926690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892c00f-a4ca-43e4-95d7-426af153b0d8 admin admin] PATH_INFO: `/v3/projects/c5212a99bb034d42907aee35a8fd498a/users/9681b021e3a64aa4a3fff56e39fd6e99/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:57.942549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892c00f-a4ca-43e4-95d7-426af153b0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5212a99bb034d42907aee35a8fd498a, user_id=9681b021e3a64aa4a3fff56e39fd6e99, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:57.946338 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892c00f-a4ca-43e4-95d7-426af153b0d8 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:57.959525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 938/1883] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:57 2022] PUT /identity/v3/projects/c5212a99bb034d42907aee35a8fd498a/users/9681b021e3a64aa4a3fff56e39fd6e99/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:57.964077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def22d65-9b1c-4cc3-a630-3e819c6cac8b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:57.987911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def22d65-9b1c-4cc3-a630-3e819c6cac8b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:58.071788 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f963353-60a8-4253-a5dd-1172f7b65f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:58.072420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f963353-60a8-4253-a5dd-1172f7b65f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:58.072697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f963353-60a8-4253-a5dd-1172f7b65f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:58.073006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f963353-60a8-4253-a5dd-1172f7b65f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:58.073792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f963353-60a8-4253-a5dd-1172f7b65f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:58.075422 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f963353-60a8-4253-a5dd-1172f7b65f1a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:58.081814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:41:58.088915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 946/1884] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:41:58.092200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892c00f-a4ca-43e4-95d7-426af153b0d8 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:58.104663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892c00f-a4ca-43e4-95d7-426af153b0d8 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:58.110330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2e6e8-533b-4cb6-858e-ac9da1b1e0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:58.111088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2e6e8-533b-4cb6-858e-ac9da1b1e0ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:58.111459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2e6e8-533b-4cb6-858e-ac9da1b1e0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:58.111821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2e6e8-533b-4cb6-858e-ac9da1b1e0ab admin admin] PATH_INFO: `/v3/projects/c5212a99bb034d42907aee35a8fd498a/users/9681b021e3a64aa4a3fff56e39fd6e99/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:58.114055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2e6e8-533b-4cb6-858e-ac9da1b1e0ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5212a99bb034d42907aee35a8fd498a, user_id=9681b021e3a64aa4a3fff56e39fd6e99, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:58.116901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2e6e8-533b-4cb6-858e-ac9da1b1e0ab admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:58.132719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 939/1885] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:41:58 2022] PUT /identity/v3/projects/c5212a99bb034d42907aee35a8fd498a/users/9681b021e3a64aa4a3fff56e39fd6e99/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:41:58.144577 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ebe3a-40ae-4e8c-8f22-3e325df5cbc8 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:58.144897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ebe3a-40ae-4e8c-8f22-3e325df5cbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:58.145168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ebe3a-40ae-4e8c-8f22-3e325df5cbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:58.238776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-363ebe3a-40ae-4e8c-8f22-3e325df5cbc8 None None] MFA Rules not processed for user `9681b021e3a64aa4a3fff56e39fd6e99`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:58.241004 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-363ebe3a-40ae-4e8c-8f22-3e325df5cbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:58.348146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 947/1886] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:41:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:58.357521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2e6e8-533b-4cb6-858e-ac9da1b1e0ab admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:58.359645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0fe2e6e8-533b-4cb6-858e-ac9da1b1e0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:58.447696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2e6e8-533b-4cb6-858e-ac9da1b1e0ab admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:58.452181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47404e-ddfb-4ab3-a39f-1d04052efcc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:58.452994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47404e-ddfb-4ab3-a39f-1d04052efcc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:58.453270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47404e-ddfb-4ab3-a39f-1d04052efcc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:58.453537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47404e-ddfb-4ab3-a39f-1d04052efcc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:58.526786 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47404e-ddfb-4ab3-a39f-1d04052efcc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:58.528229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47404e-ddfb-4ab3-a39f-1d04052efcc8 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:58.537672 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 940/1887] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:41:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:41:59.388430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96de99-c1a3-4066-be3e-77911ebaed69 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:59.389498 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96de99-c1a3-4066-be3e-77911ebaed69 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:59.389978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96de99-c1a3-4066-be3e-77911ebaed69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:59.412485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-7e96de99-c1a3-4066-be3e-77911ebaed69 None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:41:59.414027 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7e96de99-c1a3-4066-be3e-77911ebaed69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:59.486878 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 948/1888] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:41:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:41:59.540335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47404e-ddfb-4ab3-a39f-1d04052efcc8 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:41:59.542431 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-db47404e-ddfb-4ab3-a39f-1d04052efcc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:41:59.631071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47404e-ddfb-4ab3-a39f-1d04052efcc8 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:41:59.637632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961570b-c10c-49fa-a5c8-6c487a8e47a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:41:59.638430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961570b-c10c-49fa-a5c8-6c487a8e47a1 service swift] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:41:59.639089 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961570b-c10c-49fa-a5c8-6c487a8e47a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:41:59.639547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961570b-c10c-49fa-a5c8-6c487a8e47a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:41:59.650567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961570b-c10c-49fa-a5c8-6c487a8e47a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:41:59.653167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961570b-c10c-49fa-a5c8-6c487a8e47a1 service swift] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:41:59.669329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 941/1889] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:41:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:42:00.121745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96de99-c1a3-4066-be3e-77911ebaed69 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:00.123225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7e96de99-c1a3-4066-be3e-77911ebaed69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:00.146255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96de99-c1a3-4066-be3e-77911ebaed69 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:00.153072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1b731-d333-4ab1-817e-a3de71e5eb6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:00.153712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1b731-d333-4ab1-817e-a3de71e5eb6c service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:00.153990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1b731-d333-4ab1-817e-a3de71e5eb6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:00.154265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1b731-d333-4ab1-817e-a3de71e5eb6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:00.225551 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1b731-d333-4ab1-817e-a3de71e5eb6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:00.227521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1b731-d333-4ab1-817e-a3de71e5eb6c service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:00.237164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 949/1890] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:00.340627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c5f86-df5c-4b27-abda-765b6c8fcd22 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:00.341124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c5f86-df5c-4b27-abda-765b6c8fcd22 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:00.341535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c5f86-df5c-4b27-abda-765b6c8fcd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:00.362151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-cb0c5f86-df5c-4b27-abda-765b6c8fcd22 None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:00.363790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cb0c5f86-df5c-4b27-abda-765b6c8fcd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:00.375143 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 942/1891] 10.222.0.17 () {58 vars in 967 bytes} [Tue Oct 18 11:42:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:00.389113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1b731-d333-4ab1-817e-a3de71e5eb6c service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:00.401857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1b731-d333-4ab1-817e-a3de71e5eb6c service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:00.408309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1745e-2f03-41f1-ac1a-d0e17f0d0fbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:00.409121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1745e-2f03-41f1-ac1a-d0e17f0d0fbe service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:00.409526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1745e-2f03-41f1-ac1a-d0e17f0d0fbe service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:00.409786 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1745e-2f03-41f1-ac1a-d0e17f0d0fbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:00.420686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1745e-2f03-41f1-ac1a-d0e17f0d0fbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:00.422325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1745e-2f03-41f1-ac1a-d0e17f0d0fbe service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:00.431974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 950/1892] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:42:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:42:00.765187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d9f3d-e983-4aa3-87da-c54f50705beb None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:00.765653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d9f3d-e983-4aa3-87da-c54f50705beb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:00.765781 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d9f3d-e983-4aa3-87da-c54f50705beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:00.773270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:00.793350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ab5d9f3d-e983-4aa3-87da-c54f50705beb None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:00.794632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ab5d9f3d-e983-4aa3-87da-c54f50705beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:00.803602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 943/1893] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:42:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:00.817053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1745e-2f03-41f1-ac1a-d0e17f0d0fbe service swift] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:00.828201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1745e-2f03-41f1-ac1a-d0e17f0d0fbe service swift] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:00.834637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db0c51-2762-481d-9e91-a5b9ffd92dac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:00.835484 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db0c51-2762-481d-9e91-a5b9ffd92dac service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:00.836041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db0c51-2762-481d-9e91-a5b9ffd92dac service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:00.837093 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db0c51-2762-481d-9e91-a5b9ffd92dac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:00.851310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db0c51-2762-481d-9e91-a5b9ffd92dac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:00.853677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db0c51-2762-481d-9e91-a5b9ffd92dac service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:00.862727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 951/1894] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:42:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:42:00.873968 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d9f3d-e983-4aa3-87da-c54f50705beb None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:00.885892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d9f3d-e983-4aa3-87da-c54f50705beb None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:00.890465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc5775-cc9a-4912-8cce-032701f9c5bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:00.891763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc5775-cc9a-4912-8cce-032701f9c5bd service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:00.892042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc5775-cc9a-4912-8cce-032701f9c5bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:00.892340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc5775-cc9a-4912-8cce-032701f9c5bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:00.906618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc5775-cc9a-4912-8cce-032701f9c5bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:00.908253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc5775-cc9a-4912-8cce-032701f9c5bd service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:00.919061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 944/1895] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:01.272449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117443b-698d-4b22-bc35-41d4e4c739c4 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:01.272734 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117443b-698d-4b22-bc35-41d4e4c739c4 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:01.273183 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117443b-698d-4b22-bc35-41d4e4c739c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:01.280748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:01.295605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-3117443b-698d-4b22-bc35-41d4e4c739c4 None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:01.296981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3117443b-698d-4b22-bc35-41d4e4c739c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:01.306331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 952/1896] 10.222.0.17 () {58 vars in 967 bytes} [Tue Oct 18 11:42:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:01.319184 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc5775-cc9a-4912-8cce-032701f9c5bd service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:01.331230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc5775-cc9a-4912-8cce-032701f9c5bd service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:01.336143 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ac47e-fe0c-4445-a389-e08c7a7ad54b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:01.336772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ac47e-fe0c-4445-a389-e08c7a7ad54b service swift] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:01.337069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ac47e-fe0c-4445-a389-e08c7a7ad54b service swift] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:01.337337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ac47e-fe0c-4445-a389-e08c7a7ad54b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:01.346082 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ac47e-fe0c-4445-a389-e08c7a7ad54b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:01.347760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ac47e-fe0c-4445-a389-e08c7a7ad54b service swift] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:01.356336 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 945/1897] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:42:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:42:03.652091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117443b-698d-4b22-bc35-41d4e4c739c4 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:03.656142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3117443b-698d-4b22-bc35-41d4e4c739c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:03.762393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117443b-698d-4b22-bc35-41d4e4c739c4 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:03.767032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:03.767669 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:03.767950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:03.768400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] PATH_INFO: `/v3/users/f286fcc4b509485dbb958873a2207592` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:03.769577 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f286fcc4b509485dbb958873a2207592)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:03.771682 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:03.835326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f286fcc4b509485dbb958873a2207592'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:03.856892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f286fcc4b509485dbb958873a2207592'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:03.857761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f286fcc4b509485dbb958873a2207592'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:03.866127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f286fcc4b509485dbb958873a2207592'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:03.867118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f286fcc4b509485dbb958873a2207592'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:03.889068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 953/1898] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:42:03 2022] DELETE /identity/v3/users/f286fcc4b509485dbb958873a2207592 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:03.894871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669582-fa51-49da-9f2b-6f7ad1de0cf3 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:03.895440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669582-fa51-49da-9f2b-6f7ad1de0cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:03.895440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669582-fa51-49da-9f2b-6f7ad1de0cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:03.918800 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4b669582-fa51-49da-9f2b-6f7ad1de0cf3 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:03.920062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4b669582-fa51-49da-9f2b-6f7ad1de0cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:04.006168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 946/1899] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:42:04.016304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:04.017769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:04.081321 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eae47d-cf08-4c07-9abe-5aca1d0cfea9 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:04.085138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edb3f9-0216-4724-bd86-cc176f4a1385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:04.085747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edb3f9-0216-4724-bd86-cc176f4a1385 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.086051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edb3f9-0216-4724-bd86-cc176f4a1385 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.086319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edb3f9-0216-4724-bd86-cc176f4a1385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.096335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edb3f9-0216-4724-bd86-cc176f4a1385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:04.097810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edb3f9-0216-4724-bd86-cc176f4a1385 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:04.109292 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 954/1900] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:04.312687 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669582-fa51-49da-9f2b-6f7ad1de0cf3 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:04.315703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4b669582-fa51-49da-9f2b-6f7ad1de0cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:04.331339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669582-fa51-49da-9f2b-6f7ad1de0cf3 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:04.336718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:04.337647 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.337992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.338328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] PATH_INFO: `/v3/projects/4fb846ed3b9141c694927b06b387b433` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.340262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fb846ed3b9141c694927b06b387b433)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:04.343042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:04.413006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fb846ed3b9141c694927b06b387b433', 'request_id': 'req-057dd8d1-cb8f-41f5-86ff-ca325d74e969'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:04.421326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fb846ed3b9141c694927b06b387b433', 'request_id': 'req-057dd8d1-cb8f-41f5-86ff-ca325d74e969'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:04.430469 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fb846ed3b9141c694927b06b387b433', 'request_id': 'req-057dd8d1-cb8f-41f5-86ff-ca325d74e969'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:04.444062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] The token cache is being invalidate because project 4fb846ed3b9141c694927b06b387b433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:42:04.444359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:04.445108 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-057dd8d1-cb8f-41f5-86ff-ca325d74e969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:04.450920 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 947/1901] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:42:04 2022] DELETE /identity/v3/projects/4fb846ed3b9141c694927b06b387b433 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:04.454954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbb421-e3a8-470f-bd7a-ff51e53f4e40 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.455184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbb421-e3a8-470f-bd7a-ff51e53f4e40 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.455454 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbb421-e3a8-470f-bd7a-ff51e53f4e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.477636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-cbbbb421-e3a8-470f-bd7a-ff51e53f4e40 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:04.478854 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cbbbb421-e3a8-470f-bd7a-ff51e53f4e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:04.494919 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:04.565344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 955/1902] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:04.573973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce5bff-9419-4baa-a0b9-205df63dffcf None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.574353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce5bff-9419-4baa-a0b9-205df63dffcf None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.574717 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce5bff-9419-4baa-a0b9-205df63dffcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.598111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-28ce5bff-9419-4baa-a0b9-205df63dffcf None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:04.599351 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-28ce5bff-9419-4baa-a0b9-205df63dffcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:04.609624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 948/1903] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:42:04.613991 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbb421-e3a8-470f-bd7a-ff51e53f4e40 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:04.627708 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbb421-e3a8-470f-bd7a-ff51e53f4e40 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:04.633168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb70dc-e717-4b01-a1df-bfd46b22a284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:04.633568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb70dc-e717-4b01-a1df-bfd46b22a284 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.633890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb70dc-e717-4b01-a1df-bfd46b22a284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.634113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb70dc-e717-4b01-a1df-bfd46b22a284 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.634771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb70dc-e717-4b01-a1df-bfd46b22a284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:42:04.635075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb70dc-e717-4b01-a1df-bfd46b22a284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:04.636680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb70dc-e717-4b01-a1df-bfd46b22a284 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:04.650251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 956/1904] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:42:04.654510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce5bff-9419-4baa-a0b9-205df63dffcf None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:04.665755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce5bff-9419-4baa-a0b9-205df63dffcf None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:04.672208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e3d01-af23-4fe7-a843-78552cb9557c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:04.673032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e3d01-af23-4fe7-a843-78552cb9557c admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.673377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e3d01-af23-4fe7-a843-78552cb9557c admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.673713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e3d01-af23-4fe7-a843-78552cb9557c admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.674422 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e3d01-af23-4fe7-a843-78552cb9557c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:04.676979 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e3d01-af23-4fe7-a843-78552cb9557c admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:04.714390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 949/1905] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:42:04 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:04.731807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb70dc-e717-4b01-a1df-bfd46b22a284 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:04.734152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb70dc-e717-4b01-a1df-bfd46b22a284 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:04.818259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71945cd-4840-43a0-ae65-8ef01450759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:04.818958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71945cd-4840-43a0-ae65-8ef01450759a admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.819316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71945cd-4840-43a0-ae65-8ef01450759a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.819663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71945cd-4840-43a0-ae65-8ef01450759a admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.820401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71945cd-4840-43a0-ae65-8ef01450759a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:04.822412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71945cd-4840-43a0-ae65-8ef01450759a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:04.848900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 957/1906] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:42:04 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:04.850411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e3d01-af23-4fe7-a843-78552cb9557c admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:04.861547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e3d01-af23-4fe7-a843-78552cb9557c admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:04.866554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1a88-1e6f-4380-a2b6-cdb78d06e839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:04.867220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1a88-1e6f-4380-a2b6-cdb78d06e839 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.868016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1a88-1e6f-4380-a2b6-cdb78d06e839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.868288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1a88-1e6f-4380-a2b6-cdb78d06e839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.868971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1a88-1e6f-4380-a2b6-cdb78d06e839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:04.870611 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1a88-1e6f-4380-a2b6-cdb78d06e839 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:04.881259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 950/1907] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:42:04.885611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71945cd-4840-43a0-ae65-8ef01450759a admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:04.897077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71945cd-4840-43a0-ae65-8ef01450759a admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:04.902598 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855ca45-17db-4f85-8082-255c1041d7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:04.903210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ca45-17db-4f85-8082-255c1041d7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.903670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ca45-17db-4f85-8082-255c1041d7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.903943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ca45-17db-4f85-8082-255c1041d7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.904585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855ca45-17db-4f85-8082-255c1041d7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:04.906436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855ca45-17db-4f85-8082-255c1041d7e8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:04.920877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 958/1908] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:42:04.922673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1a88-1e6f-4380-a2b6-cdb78d06e839 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:04.933433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1a88-1e6f-4380-a2b6-cdb78d06e839 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:04.938050 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2d897-c856-47a4-814f-ecccc5ff900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:04.938706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2d897-c856-47a4-814f-ecccc5ff900e admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.938983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2d897-c856-47a4-814f-ecccc5ff900e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.939282 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2d897-c856-47a4-814f-ecccc5ff900e admin admin] PATH_INFO: `/v3/projects/b63f8bd1dce742b09e413947db416e66/users/b9fed14900304dc99c4c1d5fdfe204af/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.951137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:04.955417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2d897-c856-47a4-814f-ecccc5ff900e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63f8bd1dce742b09e413947db416e66, user_id=b9fed14900304dc99c4c1d5fdfe204af, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:04.958884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2d897-c856-47a4-814f-ecccc5ff900e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:04.973012 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 951/1909] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:42:04 2022] PUT /identity/v3/projects/b63f8bd1dce742b09e413947db416e66/users/b9fed14900304dc99c4c1d5fdfe204af/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:04.978180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ea2ea-c970-4255-b536-11e8ac2b8083 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:04.978474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ea2ea-c970-4255-b536-11e8ac2b8083 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:04.978746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ea2ea-c970-4255-b536-11e8ac2b8083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:04.998710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-f95ea2ea-c970-4255-b536-11e8ac2b8083 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:04.999903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f95ea2ea-c970-4255-b536-11e8ac2b8083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:05.062375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 959/1910] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:05.065998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2d897-c856-47a4-814f-ecccc5ff900e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:05.067741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-52e2d897-c856-47a4-814f-ecccc5ff900e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:05.072262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ea2ea-c970-4255-b536-11e8ac2b8083 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:05.073598 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f95ea2ea-c970-4255-b536-11e8ac2b8083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:05.081812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2d897-c856-47a4-814f-ecccc5ff900e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:05.086715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:05.087355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:05.087721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:05.087994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] PATH_INFO: `/v3/users/ed6859f1acef4c43bb500d5b9c860d23` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:05.089528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed6859f1acef4c43bb500d5b9c860d23)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:05.091439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:05.106572 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:05.144516 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed6859f1acef4c43bb500d5b9c860d23'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:05.144636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ea2ea-c970-4255-b536-11e8ac2b8083 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:05.149230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb26dd0-689d-4932-b281-76154dd84d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:05.150167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26dd0-689d-4932-b281-76154dd84d5f service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:05.150462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26dd0-689d-4932-b281-76154dd84d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:05.150862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26dd0-689d-4932-b281-76154dd84d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:05.154019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed6859f1acef4c43bb500d5b9c860d23'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:05.154830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed6859f1acef4c43bb500d5b9c860d23'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:05.162727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb26dd0-689d-4932-b281-76154dd84d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:05.169808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb26dd0-689d-4932-b281-76154dd84d5f service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:05.175478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 960/1911] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:05.185266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed6859f1acef4c43bb500d5b9c860d23'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:05.195192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed6859f1acef4c43bb500d5b9c860d23'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:05.205219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 952/1912] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:42:05 2022] DELETE /identity/v3/users/ed6859f1acef4c43bb500d5b9c860d23 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:05.209539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be0820-8820-41d0-a27d-bb4f1fa73c17 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:05.209827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be0820-8820-41d0-a27d-bb4f1fa73c17 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:05.210102 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be0820-8820-41d0-a27d-bb4f1fa73c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:05.230090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-48be0820-8820-41d0-a27d-bb4f1fa73c17 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:05.231233 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-48be0820-8820-41d0-a27d-bb4f1fa73c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:05.309570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 961/1913] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:05.319716 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:05.321765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:05.385852 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b81bb2-aa0c-47dc-b504-97a2f76a06ec admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:05.389777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf8854-f340-46cd-84df-a72cf6460466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:05.390407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf8854-f340-46cd-84df-a72cf6460466 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:05.390684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf8854-f340-46cd-84df-a72cf6460466 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:05.390958 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf8854-f340-46cd-84df-a72cf6460466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:05.400631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf8854-f340-46cd-84df-a72cf6460466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:05.402123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf8854-f340-46cd-84df-a72cf6460466 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:05.412566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 953/1914] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:05.988664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be0820-8820-41d0-a27d-bb4f1fa73c17 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:05.994426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-48be0820-8820-41d0-a27d-bb4f1fa73c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:06.007297 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be0820-8820-41d0-a27d-bb4f1fa73c17 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:06.017562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:06.019242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:06.019777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:06.021052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] PATH_INFO: `/v3/projects/34ea47b80fc840458abccbec052ab5e6` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:06.025241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ea47b80fc840458abccbec052ab5e6)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:06.027253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:06.112784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ea47b80fc840458abccbec052ab5e6', 'request_id': 'req-5e9a2762-b763-4a54-8d79-df9f42c2fad5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:06.120340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ea47b80fc840458abccbec052ab5e6', 'request_id': 'req-5e9a2762-b763-4a54-8d79-df9f42c2fad5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:06.132516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ea47b80fc840458abccbec052ab5e6', 'request_id': 'req-5e9a2762-b763-4a54-8d79-df9f42c2fad5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:06.150129 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] The token cache is being invalidate because project 34ea47b80fc840458abccbec052ab5e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:42:06.150481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:06.151171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:06.153169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 962/1915] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:42:05 2022] DELETE /identity/v3/projects/34ea47b80fc840458abccbec052ab5e6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:06.159930 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf8854-f340-46cd-84df-a72cf6460466 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:06.196334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf8854-f340-46cd-84df-a72cf6460466 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:06.283869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:06.284542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:06.284935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:06.285521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] PATH_INFO: `/v3/users/901ae59ca2fa47abab8a951888566360` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:06.286777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=901ae59ca2fa47abab8a951888566360)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:06.289559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:06.295316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:06.355017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '901ae59ca2fa47abab8a951888566360'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:06.362954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '901ae59ca2fa47abab8a951888566360'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:06.363764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '901ae59ca2fa47abab8a951888566360'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:06.380668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '901ae59ca2fa47abab8a951888566360'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:06.391034 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '901ae59ca2fa47abab8a951888566360'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:06.399920 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 954/1916] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:42:06 2022] DELETE /identity/v3/users/901ae59ca2fa47abab8a951888566360 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:06.761285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:06.762882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:06.781886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a2762-b763-4a54-8d79-df9f42c2fad5 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:06.878917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:06.908198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:06.909343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:06.909950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:06.910548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] PATH_INFO: `/v3/projects/035ffd1e912a4a16b3406d80434baf35` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:06.912226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=035ffd1e912a4a16b3406d80434baf35)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:06.914752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:06.995871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '035ffd1e912a4a16b3406d80434baf35', 'request_id': 'req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:07.023205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '035ffd1e912a4a16b3406d80434baf35', 'request_id': 'req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:07.034498 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '035ffd1e912a4a16b3406d80434baf35', 'request_id': 'req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:07.053469 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] The token cache is being invalidate because project 035ffd1e912a4a16b3406d80434baf35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:42:07.053931 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:07.056055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:07.061580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 963/1917] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:42:06 2022] DELETE /identity/v3/projects/035ffd1e912a4a16b3406d80434baf35 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:07.062465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:07.065722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:07.080023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699761b-11e9-4336-8e90-a961d57c17f1 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:07.142983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:07.143644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:07.143920 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:07.144190 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] PATH_INFO: `/v3/users/9681b021e3a64aa4a3fff56e39fd6e99` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:07.145445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9681b021e3a64aa4a3fff56e39fd6e99)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:07.147684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:07.242859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9681b021e3a64aa4a3fff56e39fd6e99'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:07.265279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9681b021e3a64aa4a3fff56e39fd6e99'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:07.266350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9681b021e3a64aa4a3fff56e39fd6e99'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:07.281768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9681b021e3a64aa4a3fff56e39fd6e99'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:07.296736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c3348ab4-7bf3-4af2-bbee-7a2cb0c0b3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9681b021e3a64aa4a3fff56e39fd6e99'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:07.310566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 955/1918] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:42:07 2022] DELETE /identity/v3/users/9681b021e3a64aa4a3fff56e39fd6e99 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:42:08.112357 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:08.114513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:08.142615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cd91a-b41c-4ec2-9d72-fcbf72a06c0e admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:08.232914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:08.233853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:08.234279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:08.234696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] PATH_INFO: `/v3/projects/c5212a99bb034d42907aee35a8fd498a` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:08.236010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5212a99bb034d42907aee35a8fd498a)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:08.238132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:08.316957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5212a99bb034d42907aee35a8fd498a', 'request_id': 'req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:08.330048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5212a99bb034d42907aee35a8fd498a', 'request_id': 'req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:08.345458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5212a99bb034d42907aee35a8fd498a', 'request_id': 'req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:08.362686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] The token cache is being invalidate because project c5212a99bb034d42907aee35a8fd498a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:42:08.363144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:08.363854 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-97c9ddaf-ddd6-4e79-89f0-ca52fc94c44d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:08.369957 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 964/1919] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:42:08 2022] DELETE /identity/v3/projects/c5212a99bb034d42907aee35a8fd498a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:08.371728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668c9d1-d6eb-483b-9d0c-8d59f73fabcb None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:08.372187 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668c9d1-d6eb-483b-9d0c-8d59f73fabcb None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:08.372347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668c9d1-d6eb-483b-9d0c-8d59f73fabcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:08.381771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:08.399939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9668c9d1-d6eb-483b-9d0c-8d59f73fabcb None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:08.402059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9668c9d1-d6eb-483b-9d0c-8d59f73fabcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:08.548635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 956/1920] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:08.549296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22ee46-c911-46ed-a17f-9677bf9edd66 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:08.549958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22ee46-c911-46ed-a17f-9677bf9edd66 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:08.550378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22ee46-c911-46ed-a17f-9677bf9edd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:08.585020 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1a22ee46-c911-46ed-a17f-9677bf9edd66 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:08.587167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1a22ee46-c911-46ed-a17f-9677bf9edd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:08.623966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 965/1921] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:42:08.627435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668c9d1-d6eb-483b-9d0c-8d59f73fabcb None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:08.674442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668c9d1-d6eb-483b-9d0c-8d59f73fabcb None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:08.682634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430dfc8-6653-4e93-bb04-9a911abc739a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:08.683448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430dfc8-6653-4e93-bb04-9a911abc739a admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:08.683762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430dfc8-6653-4e93-bb04-9a911abc739a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:08.684132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430dfc8-6653-4e93-bb04-9a911abc739a admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:08.685102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430dfc8-6653-4e93-bb04-9a911abc739a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:42:08.685309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430dfc8-6653-4e93-bb04-9a911abc739a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:08.688873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430dfc8-6653-4e93-bb04-9a911abc739a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:08.708452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 957/1922] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:42:08.710750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22ee46-c911-46ed-a17f-9677bf9edd66 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:08.716713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:08.722419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22ee46-c911-46ed-a17f-9677bf9edd66 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:08.733096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7410f-5379-4a15-b955-46b9907a27d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:08.733741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7410f-5379-4a15-b955-46b9907a27d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:08.734070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7410f-5379-4a15-b955-46b9907a27d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:08.734395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7410f-5379-4a15-b955-46b9907a27d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:08.735034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7410f-5379-4a15-b955-46b9907a27d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:08.739758 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7410f-5379-4a15-b955-46b9907a27d3 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:08.798951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 966/1923] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:42:08 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:08.801213 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430dfc8-6653-4e93-bb04-9a911abc739a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:08.814036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430dfc8-6653-4e93-bb04-9a911abc739a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:08.956609 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80f94-6713-49b6-8280-414a869bd910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:08.958062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80f94-6713-49b6-8280-414a869bd910 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:08.958837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80f94-6713-49b6-8280-414a869bd910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:08.961233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80f94-6713-49b6-8280-414a869bd910 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:08.964252 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80f94-6713-49b6-8280-414a869bd910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:08.968775 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80f94-6713-49b6-8280-414a869bd910 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:09.017831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 958/1924] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:42:08 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:09.022239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7410f-5379-4a15-b955-46b9907a27d3 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:09.047271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7410f-5379-4a15-b955-46b9907a27d3 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:09.054140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a154a1d-2881-4d63-8080-ccdbfc7d033b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:09.055060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a154a1d-2881-4d63-8080-ccdbfc7d033b admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:09.055615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a154a1d-2881-4d63-8080-ccdbfc7d033b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:09.056061 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a154a1d-2881-4d63-8080-ccdbfc7d033b admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:09.057022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a154a1d-2881-4d63-8080-ccdbfc7d033b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:09.058819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a154a1d-2881-4d63-8080-ccdbfc7d033b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:09.071392 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 967/1925] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:42:09.075360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80f94-6713-49b6-8280-414a869bd910 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:09.093016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80f94-6713-49b6-8280-414a869bd910 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:09.107689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a42c67-1107-455d-af05-c8ac5044997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:09.108247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a42c67-1107-455d-af05-c8ac5044997b admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:09.108519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a42c67-1107-455d-af05-c8ac5044997b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:09.108786 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a42c67-1107-455d-af05-c8ac5044997b admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:09.109462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a42c67-1107-455d-af05-c8ac5044997b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:09.111095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a42c67-1107-455d-af05-c8ac5044997b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:09.133772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 959/1926] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:42:09.135288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a154a1d-2881-4d63-8080-ccdbfc7d033b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:09.149691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a154a1d-2881-4d63-8080-ccdbfc7d033b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:09.162114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f69c05-b3d9-45ab-a51d-bfb59726fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:09.163099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f69c05-b3d9-45ab-a51d-bfb59726fca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:09.164071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f69c05-b3d9-45ab-a51d-bfb59726fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:09.164583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f69c05-b3d9-45ab-a51d-bfb59726fca9 admin admin] PATH_INFO: `/v3/projects/113075c27b264ae9a69483d4969ec030/users/3d1c84c2aae84e41af4e069a2f97ef07/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:09.193807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f69c05-b3d9-45ab-a51d-bfb59726fca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=113075c27b264ae9a69483d4969ec030, user_id=3d1c84c2aae84e41af4e069a2f97ef07, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:09.197129 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f69c05-b3d9-45ab-a51d-bfb59726fca9 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:09.219561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 968/1927] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:42:09 2022] PUT /identity/v3/projects/113075c27b264ae9a69483d4969ec030/users/3d1c84c2aae84e41af4e069a2f97ef07/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:09.231244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab849438-ac0b-4047-b194-93189a22a5ca None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:09.231742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab849438-ac0b-4047-b194-93189a22a5ca None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:09.232195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab849438-ac0b-4047-b194-93189a22a5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:09.268135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ab849438-ac0b-4047-b194-93189a22a5ca None None] MFA Rules not processed for user `3d1c84c2aae84e41af4e069a2f97ef07`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:09.281772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ab849438-ac0b-4047-b194-93189a22a5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:09.386113 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:09.397058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 960/1928] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:09.409795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f69c05-b3d9-45ab-a51d-bfb59726fca9 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:09.414942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-47f69c05-b3d9-45ab-a51d-bfb59726fca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:09.491787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f69c05-b3d9-45ab-a51d-bfb59726fca9 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:09.497867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cf18b-f00a-4848-9538-701025568dbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:09.499766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cf18b-f00a-4848-9538-701025568dbf service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:09.500741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cf18b-f00a-4848-9538-701025568dbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:09.501340 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cf18b-f00a-4848-9538-701025568dbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:09.515024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cf18b-f00a-4848-9538-701025568dbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:09.517549 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cf18b-f00a-4848-9538-701025568dbf service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:09.528675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 969/1929] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:11.920202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f76ebf-8c04-4728-82a4-86de1c20f63d None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:11.921241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f76ebf-8c04-4728-82a4-86de1c20f63d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:11.921722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f76ebf-8c04-4728-82a4-86de1c20f63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:11.959763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-10f76ebf-8c04-4728-82a4-86de1c20f63d None None] MFA Rules not processed for user `b9fed14900304dc99c4c1d5fdfe204af`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:11.961139 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-10f76ebf-8c04-4728-82a4-86de1c20f63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:12.035888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 961/1930] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:12.040464 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cf18b-f00a-4848-9538-701025568dbf service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:12.041949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3c1cf18b-f00a-4848-9538-701025568dbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:12.112892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cf18b-f00a-4848-9538-701025568dbf service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:12.118600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f5863-8cbb-43db-9997-2e221c7d8105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:12.119062 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f5863-8cbb-43db-9997-2e221c7d8105 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:12.119338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f5863-8cbb-43db-9997-2e221c7d8105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:12.119703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f5863-8cbb-43db-9997-2e221c7d8105 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:12.120248 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f5863-8cbb-43db-9997-2e221c7d8105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:12.122197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f5863-8cbb-43db-9997-2e221c7d8105 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:12.151094 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 970/1931] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:42:12 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:12.155347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f76ebf-8c04-4728-82a4-86de1c20f63d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:12.167467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f76ebf-8c04-4728-82a4-86de1c20f63d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:12.231070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f15a3-08b1-4aa6-9210-735c31cb07ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:12.231724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f15a3-08b1-4aa6-9210-735c31cb07ea admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:12.232003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f15a3-08b1-4aa6-9210-735c31cb07ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:12.232276 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f15a3-08b1-4aa6-9210-735c31cb07ea admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:12.232977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f15a3-08b1-4aa6-9210-735c31cb07ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:12.235065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f15a3-08b1-4aa6-9210-735c31cb07ea admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:12.260803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 962/1932] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:42:12 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:12.262239 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f5863-8cbb-43db-9997-2e221c7d8105 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:12.272508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f5863-8cbb-43db-9997-2e221c7d8105 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:12.277157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad91e4-29bb-4ee1-8e70-76dbe7e2f55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:12.277754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad91e4-29bb-4ee1-8e70-76dbe7e2f55c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:12.278029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad91e4-29bb-4ee1-8e70-76dbe7e2f55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:12.278301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad91e4-29bb-4ee1-8e70-76dbe7e2f55c admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:12.279007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad91e4-29bb-4ee1-8e70-76dbe7e2f55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:12.280583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad91e4-29bb-4ee1-8e70-76dbe7e2f55c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:12.291637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 971/1933] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:42:12.295869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f15a3-08b1-4aa6-9210-735c31cb07ea admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:12.306980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f15a3-08b1-4aa6-9210-735c31cb07ea admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:12.311663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb701b1-f22a-49a4-bef5-76bf57584d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:12.312365 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb701b1-f22a-49a4-bef5-76bf57584d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:12.312647 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb701b1-f22a-49a4-bef5-76bf57584d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:12.312949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb701b1-f22a-49a4-bef5-76bf57584d6c admin admin] PATH_INFO: `/v3/projects/657b38c99c63453b929ae9d2adb05d1f/users/399bb0de89ce4b1d828406683cad9672/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:12.329550 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb701b1-f22a-49a4-bef5-76bf57584d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=657b38c99c63453b929ae9d2adb05d1f, user_id=399bb0de89ce4b1d828406683cad9672, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:12.332242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb701b1-f22a-49a4-bef5-76bf57584d6c admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:12.353297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 963/1934] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:42:12 2022] PUT /identity/v3/projects/657b38c99c63453b929ae9d2adb05d1f/users/399bb0de89ce4b1d828406683cad9672/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:12.354853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad91e4-29bb-4ee1-8e70-76dbe7e2f55c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:12.359240 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb701b1-f22a-49a4-bef5-76bf57584d6c admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:12.373114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad91e4-29bb-4ee1-8e70-76dbe7e2f55c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:12.416692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:12.436511 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:12.453372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb701b1-f22a-49a4-bef5-76bf57584d6c admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:12.467409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:12.468063 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:12.468337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:12.468605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] PATH_INFO: `/v3/users/3d1c84c2aae84e41af4e069a2f97ef07` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:12.469808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d1c84c2aae84e41af4e069a2f97ef07)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:12.472727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:12.504586 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5e26f-3226-4a47-b84e-6efc3290d20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:12.505719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5e26f-3226-4a47-b84e-6efc3290d20b admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:12.506009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5e26f-3226-4a47-b84e-6efc3290d20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:12.506703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5e26f-3226-4a47-b84e-6efc3290d20b admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:12.507032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5e26f-3226-4a47-b84e-6efc3290d20b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:12.508721 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5e26f-3226-4a47-b84e-6efc3290d20b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:12.520931 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 972/1935] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:42:12.526324 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5e26f-3226-4a47-b84e-6efc3290d20b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:12.538141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d1c84c2aae84e41af4e069a2f97ef07'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:12.544374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5e26f-3226-4a47-b84e-6efc3290d20b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:12.549656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d1c84c2aae84e41af4e069a2f97ef07'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:12.550620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d1c84c2aae84e41af4e069a2f97ef07'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:12.552166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0871e3-17d4-4745-aa6e-933aa778144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:12.553173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0871e3-17d4-4745-aa6e-933aa778144a admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:12.553515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0871e3-17d4-4745-aa6e-933aa778144a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:12.554120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0871e3-17d4-4745-aa6e-933aa778144a admin admin] PATH_INFO: `/v3/projects/657b38c99c63453b929ae9d2adb05d1f/users/399bb0de89ce4b1d828406683cad9672/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:12.561177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0871e3-17d4-4745-aa6e-933aa778144a admin admin] RBAC: Authorizing `identity:create_grant(project_id=657b38c99c63453b929ae9d2adb05d1f, user_id=399bb0de89ce4b1d828406683cad9672, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:12.564391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0871e3-17d4-4745-aa6e-933aa778144a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:12.566969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d1c84c2aae84e41af4e069a2f97ef07'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:12.579644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d1c84c2aae84e41af4e069a2f97ef07'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:12.583110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 973/1936] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:42:12 2022] PUT /identity/v3/projects/657b38c99c63453b929ae9d2adb05d1f/users/399bb0de89ce4b1d828406683cad9672/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:42:12.588529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 964/1937] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:42:12 2022] DELETE /identity/v3/users/3d1c84c2aae84e41af4e069a2f97ef07 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:12.592634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8faad-6526-43ab-b988-2d84a9b7ad7a None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:12.593148 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8faad-6526-43ab-b988-2d84a9b7ad7a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:12.593863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8faad-6526-43ab-b988-2d84a9b7ad7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:12.639115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-16c8faad-6526-43ab-b988-2d84a9b7ad7a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:12.640391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-16c8faad-6526-43ab-b988-2d84a9b7ad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:12.722837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 974/1938] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:42:12.733547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:12.735417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:12.814120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb46b8-cd31-4347-8079-df0425548a9e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:12.818026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1b9a0-18c7-47ad-a262-ddc6963163bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:12.818652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1b9a0-18c7-47ad-a262-ddc6963163bb service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:12.818934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1b9a0-18c7-47ad-a262-ddc6963163bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:12.819327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1b9a0-18c7-47ad-a262-ddc6963163bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:12.846225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1b9a0-18c7-47ad-a262-ddc6963163bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:12.850569 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1b9a0-18c7-47ad-a262-ddc6963163bb service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:12.874289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 965/1939] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:13.994720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8faad-6526-43ab-b988-2d84a9b7ad7a None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:13.996903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-16c8faad-6526-43ab-b988-2d84a9b7ad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:14.014376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8faad-6526-43ab-b988-2d84a9b7ad7a None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:14.044063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:14.044747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:14.045096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:14.045364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] PATH_INFO: `/v3/projects/113075c27b264ae9a69483d4969ec030` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:14.047470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=113075c27b264ae9a69483d4969ec030)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:14.050439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:14.144875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '113075c27b264ae9a69483d4969ec030', 'request_id': 'req-87cccf3c-ec8d-4674-bca7-f906c7438825'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:14.153057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '113075c27b264ae9a69483d4969ec030', 'request_id': 'req-87cccf3c-ec8d-4674-bca7-f906c7438825'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:14.171462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '113075c27b264ae9a69483d4969ec030', 'request_id': 'req-87cccf3c-ec8d-4674-bca7-f906c7438825'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:14.188048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] The token cache is being invalidate because project 113075c27b264ae9a69483d4969ec030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:42:14.188343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:14.189529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-87cccf3c-ec8d-4674-bca7-f906c7438825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:14.191621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 975/1940] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:42:13 2022] DELETE /identity/v3/projects/113075c27b264ae9a69483d4969ec030 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:14.197301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da98e6-5336-459c-aa30-55e8426b8ad8 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:14.197804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da98e6-5336-459c-aa30-55e8426b8ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:14.198056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da98e6-5336-459c-aa30-55e8426b8ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:14.242741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-15da98e6-5336-459c-aa30-55e8426b8ad8 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:14.244092 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-15da98e6-5336-459c-aa30-55e8426b8ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:14.415257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 966/1941] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:14.424264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d197a7-8499-403a-9dc8-afd6b2fe7905 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:14.424745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d197a7-8499-403a-9dc8-afd6b2fe7905 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:14.425132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d197a7-8499-403a-9dc8-afd6b2fe7905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:14.471374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d9d197a7-8499-403a-9dc8-afd6b2fe7905 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:14.472807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d9d197a7-8499-403a-9dc8-afd6b2fe7905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:14.492078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 976/1942] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:14.492629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da98e6-5336-459c-aa30-55e8426b8ad8 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:14.509601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da98e6-5336-459c-aa30-55e8426b8ad8 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:14.528700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51d77a-a997-41f6-8bad-dcfe43b3981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:14.530761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51d77a-a997-41f6-8bad-dcfe43b3981e admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:14.531062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51d77a-a997-41f6-8bad-dcfe43b3981e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:14.531339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51d77a-a997-41f6-8bad-dcfe43b3981e admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:14.533505 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51d77a-a997-41f6-8bad-dcfe43b3981e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:42:14.533821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51d77a-a997-41f6-8bad-dcfe43b3981e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:14.536809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51d77a-a997-41f6-8bad-dcfe43b3981e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:14.556993 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 967/1943] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:42:14.561529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d197a7-8499-403a-9dc8-afd6b2fe7905 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:14.576647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d197a7-8499-403a-9dc8-afd6b2fe7905 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:14.603547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd66be-cf0c-4b9a-a0b2-04f5ef2f1507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:14.604555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd66be-cf0c-4b9a-a0b2-04f5ef2f1507 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:14.604739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd66be-cf0c-4b9a-a0b2-04f5ef2f1507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:14.605068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd66be-cf0c-4b9a-a0b2-04f5ef2f1507 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:14.606294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd66be-cf0c-4b9a-a0b2-04f5ef2f1507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:14.620822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd66be-cf0c-4b9a-a0b2-04f5ef2f1507 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:14.697074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 977/1944] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:42:14 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:14.709857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51d77a-a997-41f6-8bad-dcfe43b3981e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:14.717375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:14.731229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51d77a-a997-41f6-8bad-dcfe43b3981e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:14.871063 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd8c84-f0e0-4bc5-b67d-31567dae8cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:14.873188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd8c84-f0e0-4bc5-b67d-31567dae8cee admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:14.875267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd8c84-f0e0-4bc5-b67d-31567dae8cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:14.875808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd8c84-f0e0-4bc5-b67d-31567dae8cee admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:14.876650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd8c84-f0e0-4bc5-b67d-31567dae8cee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:14.883261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd8c84-f0e0-4bc5-b67d-31567dae8cee admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:14.939746 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 968/1945] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:42:14 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:14.943302 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd66be-cf0c-4b9a-a0b2-04f5ef2f1507 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:14.950480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:14.961378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd66be-cf0c-4b9a-a0b2-04f5ef2f1507 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:14.983765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba6941-a29f-4204-8274-7cc1be136bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:14.983765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba6941-a29f-4204-8274-7cc1be136bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:14.983765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba6941-a29f-4204-8274-7cc1be136bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:14.983765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba6941-a29f-4204-8274-7cc1be136bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:14.984119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba6941-a29f-4204-8274-7cc1be136bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:14.989764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba6941-a29f-4204-8274-7cc1be136bc6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:15.018110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 978/1946] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:42:15.021839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd8c84-f0e0-4bc5-b67d-31567dae8cee admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:15.033829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd8c84-f0e0-4bc5-b67d-31567dae8cee admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:15.039640 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40973336-64c6-4197-b3a2-896c8888d439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:15.040281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40973336-64c6-4197-b3a2-896c8888d439 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:15.040558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40973336-64c6-4197-b3a2-896c8888d439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:15.040938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40973336-64c6-4197-b3a2-896c8888d439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:15.041610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40973336-64c6-4197-b3a2-896c8888d439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:15.043241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40973336-64c6-4197-b3a2-896c8888d439 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:15.060586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 969/1947] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:42:15.064772 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba6941-a29f-4204-8274-7cc1be136bc6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:15.076504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba6941-a29f-4204-8274-7cc1be136bc6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:15.081976 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9324f78-4da8-432d-a895-45c8d94fdfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:15.082929 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9324f78-4da8-432d-a895-45c8d94fdfc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:15.083447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9324f78-4da8-432d-a895-45c8d94fdfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:15.083954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9324f78-4da8-432d-a895-45c8d94fdfc3 admin admin] PATH_INFO: `/v3/projects/66147f5ce9d14e51a54c1a6c0302a92b/users/a2bfaba865cb41cebaec1482629209a5/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:15.108319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9324f78-4da8-432d-a895-45c8d94fdfc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66147f5ce9d14e51a54c1a6c0302a92b, user_id=a2bfaba865cb41cebaec1482629209a5, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:15.111669 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9324f78-4da8-432d-a895-45c8d94fdfc3 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:15.142100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 979/1948] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:42:15 2022] PUT /identity/v3/projects/66147f5ce9d14e51a54c1a6c0302a92b/users/a2bfaba865cb41cebaec1482629209a5/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:15.152729 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74ce80-ae5e-4bd2-85dd-f70e34951e28 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:15.153188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74ce80-ae5e-4bd2-85dd-f70e34951e28 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:15.153849 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74ce80-ae5e-4bd2-85dd-f70e34951e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:15.238342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9f74ce80-ae5e-4bd2-85dd-f70e34951e28 None None] MFA Rules not processed for user `a2bfaba865cb41cebaec1482629209a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:15.265057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9f74ce80-ae5e-4bd2-85dd-f70e34951e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:15.392312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 970/1949] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:15.403854 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9324f78-4da8-432d-a895-45c8d94fdfc3 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:15.406218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b9324f78-4da8-432d-a895-45c8d94fdfc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:15.488390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9324f78-4da8-432d-a895-45c8d94fdfc3 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:15.493674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0312abe-5e09-4d9b-b6a2-a3f3bb2d3cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:15.495306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0312abe-5e09-4d9b-b6a2-a3f3bb2d3cea service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:15.495607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0312abe-5e09-4d9b-b6a2-a3f3bb2d3cea service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:15.496001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0312abe-5e09-4d9b-b6a2-a3f3bb2d3cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:15.509872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0312abe-5e09-4d9b-b6a2-a3f3bb2d3cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:15.511454 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0312abe-5e09-4d9b-b6a2-a3f3bb2d3cea service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:15.524448 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 980/1950] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:15.553561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74ce80-ae5e-4bd2-85dd-f70e34951e28 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:15.638624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74ce80-ae5e-4bd2-85dd-f70e34951e28 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:15.644056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:15.644881 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:15.645411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:15.645902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] PATH_INFO: `/v3/users/a2bfaba865cb41cebaec1482629209a5` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:15.647350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2bfaba865cb41cebaec1482629209a5)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:15.650191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:15.670943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:15.729828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2bfaba865cb41cebaec1482629209a5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:15.748000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2bfaba865cb41cebaec1482629209a5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:15.748405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2bfaba865cb41cebaec1482629209a5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:15.768940 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2bfaba865cb41cebaec1482629209a5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:15.782454 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2bfaba865cb41cebaec1482629209a5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:15.794156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 971/1951] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:42:15 2022] DELETE /identity/v3/users/a2bfaba865cb41cebaec1482629209a5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:15.798939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f2694-b36b-43b2-9067-0e6a7ee92d29 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:15.799350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f2694-b36b-43b2-9067-0e6a7ee92d29 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:15.799748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f2694-b36b-43b2-9067-0e6a7ee92d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:15.828198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d16f2694-b36b-43b2-9067-0e6a7ee92d29 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:15.830725 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d16f2694-b36b-43b2-9067-0e6a7ee92d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:15.958134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 981/1952] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:15.973392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:15.975179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:16.077951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a564d4-6e6a-4fe7-90fc-68d56a2db8ed admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:16.082105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd1e30-c1cf-49c6-980a-c6852ae2e494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:16.082744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd1e30-c1cf-49c6-980a-c6852ae2e494 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:16.083611 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd1e30-c1cf-49c6-980a-c6852ae2e494 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:16.083611 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd1e30-c1cf-49c6-980a-c6852ae2e494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:16.109159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd1e30-c1cf-49c6-980a-c6852ae2e494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:16.110631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd1e30-c1cf-49c6-980a-c6852ae2e494 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:16.133117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 972/1953] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:16.910691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f2694-b36b-43b2-9067-0e6a7ee92d29 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:16.912791 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d16f2694-b36b-43b2-9067-0e6a7ee92d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:16.934008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f2694-b36b-43b2-9067-0e6a7ee92d29 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:16.948145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:16.948827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:16.949134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:16.949413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] PATH_INFO: `/v3/projects/66147f5ce9d14e51a54c1a6c0302a92b` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:16.953263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66147f5ce9d14e51a54c1a6c0302a92b)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:16.972079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:17.077973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66147f5ce9d14e51a54c1a6c0302a92b', 'request_id': 'req-0bb015a8-673b-4267-89a4-e22a81632f26'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:17.094374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66147f5ce9d14e51a54c1a6c0302a92b', 'request_id': 'req-0bb015a8-673b-4267-89a4-e22a81632f26'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:17.118789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66147f5ce9d14e51a54c1a6c0302a92b', 'request_id': 'req-0bb015a8-673b-4267-89a4-e22a81632f26'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:17.121237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:17.148714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] The token cache is being invalidate because project 66147f5ce9d14e51a54c1a6c0302a92b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:42:17.150347 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:17.153151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0bb015a8-673b-4267-89a4-e22a81632f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:42:17.162221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 982/1954] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:42:16 2022] DELETE /identity/v3/projects/66147f5ce9d14e51a54c1a6c0302a92b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:17.164037 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08ead0-3d8b-40ce-a5f1-206acc1012f9 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:17.165064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08ead0-3d8b-40ce-a5f1-206acc1012f9 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:17.165395 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08ead0-3d8b-40ce-a5f1-206acc1012f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:17.205371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-0b08ead0-3d8b-40ce-a5f1-206acc1012f9 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:17.207413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0b08ead0-3d8b-40ce-a5f1-206acc1012f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:17.317793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 973/1955] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:42:17.326064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa170744-6e6f-4eb1-b62f-212359dffdcb None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:17.326757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa170744-6e6f-4eb1-b62f-212359dffdcb None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:17.327507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa170744-6e6f-4eb1-b62f-212359dffdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:17.363144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-fa170744-6e6f-4eb1-b62f-212359dffdcb None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:17.365250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-fa170744-6e6f-4eb1-b62f-212359dffdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:17.390156 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 983/1956] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:17.396671 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08ead0-3d8b-40ce-a5f1-206acc1012f9 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:17.412757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08ead0-3d8b-40ce-a5f1-206acc1012f9 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:17.430126 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003aacf-d858-486f-b712-02d1d5d943ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:17.431217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003aacf-d858-486f-b712-02d1d5d943ed admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:17.431217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003aacf-d858-486f-b712-02d1d5d943ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:17.431403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003aacf-d858-486f-b712-02d1d5d943ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:17.432089 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003aacf-d858-486f-b712-02d1d5d943ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:42:17.432384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003aacf-d858-486f-b712-02d1d5d943ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:17.433910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003aacf-d858-486f-b712-02d1d5d943ed admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:17.467803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 974/1957] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:42:17.469062 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa170744-6e6f-4eb1-b62f-212359dffdcb None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:17.480895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa170744-6e6f-4eb1-b62f-212359dffdcb None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:17.494584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bfbba-8865-4aae-bf83-f160b5ebb554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:17.495350 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bfbba-8865-4aae-bf83-f160b5ebb554 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:17.497301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bfbba-8865-4aae-bf83-f160b5ebb554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:17.497301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bfbba-8865-4aae-bf83-f160b5ebb554 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:17.497301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bfbba-8865-4aae-bf83-f160b5ebb554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:17.499361 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bfbba-8865-4aae-bf83-f160b5ebb554 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:17.554832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 984/1958] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:42:17 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:17.558495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003aacf-d858-486f-b712-02d1d5d943ed admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:17.571758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003aacf-d858-486f-b712-02d1d5d943ed admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:17.596067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:17.684356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaff983-0f65-451c-9c71-786fb77e4702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:17.686563 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaff983-0f65-451c-9c71-786fb77e4702 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:17.686997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaff983-0f65-451c-9c71-786fb77e4702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:17.687421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaff983-0f65-451c-9c71-786fb77e4702 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:17.688226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaff983-0f65-451c-9c71-786fb77e4702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:17.690605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaff983-0f65-451c-9c71-786fb77e4702 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:17.718760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 975/1959] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:42:17 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:17.722629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bfbba-8865-4aae-bf83-f160b5ebb554 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:17.734022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bfbba-8865-4aae-bf83-f160b5ebb554 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:17.739023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184da6a-0504-4c4a-bf83-5297e23f155b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:17.739707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184da6a-0504-4c4a-bf83-5297e23f155b admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:17.740047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184da6a-0504-4c4a-bf83-5297e23f155b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:17.740379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184da6a-0504-4c4a-bf83-5297e23f155b admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:17.741121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184da6a-0504-4c4a-bf83-5297e23f155b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:17.743541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184da6a-0504-4c4a-bf83-5297e23f155b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:17.759419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaff983-0f65-451c-9c71-786fb77e4702 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:17.759897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 985/1960] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:42:17.773728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaff983-0f65-451c-9c71-786fb77e4702 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:17.779057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d49740-890b-4a7b-a646-4c1f4cac87ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:17.779796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d49740-890b-4a7b-a646-4c1f4cac87ba admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:17.780177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d49740-890b-4a7b-a646-4c1f4cac87ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:17.780539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d49740-890b-4a7b-a646-4c1f4cac87ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:17.781338 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d49740-890b-4a7b-a646-4c1f4cac87ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:17.783043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d49740-890b-4a7b-a646-4c1f4cac87ba admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:17.800009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184da6a-0504-4c4a-bf83-5297e23f155b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:17.810655 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 976/1961] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:42:17.811696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184da6a-0504-4c4a-bf83-5297e23f155b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:17.822888 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092a31-c36b-4758-a5c6-4bf754a879ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:17.823698 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092a31-c36b-4758-a5c6-4bf754a879ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:17.824037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092a31-c36b-4758-a5c6-4bf754a879ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:17.824384 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092a31-c36b-4758-a5c6-4bf754a879ab admin admin] PATH_INFO: `/v3/projects/34d789b867f24711a6c1b5589a573560/users/f76d70b021f44d1391653477c65f95af/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:17.852731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc092a31-c36b-4758-a5c6-4bf754a879ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d789b867f24711a6c1b5589a573560, user_id=f76d70b021f44d1391653477c65f95af, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:17.855546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc092a31-c36b-4758-a5c6-4bf754a879ab admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:17.903750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 986/1962] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:42:17 2022] PUT /identity/v3/projects/34d789b867f24711a6c1b5589a573560/users/f76d70b021f44d1391653477c65f95af/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:17.910138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430347b-bd95-42c9-a834-9c1cee8d3c04 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:17.910957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430347b-bd95-42c9-a834-9c1cee8d3c04 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:17.911421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430347b-bd95-42c9-a834-9c1cee8d3c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:17.953418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3430347b-bd95-42c9-a834-9c1cee8d3c04 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:17.955245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3430347b-bd95-42c9-a834-9c1cee8d3c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:18.122283 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 977/1963] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:18.129429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092a31-c36b-4758-a5c6-4bf754a879ab admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:18.131548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-fc092a31-c36b-4758-a5c6-4bf754a879ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:18.213811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092a31-c36b-4758-a5c6-4bf754a879ab admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:18.218355 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90239a-7b99-4f68-a60c-b72ae424c655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:18.219412 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90239a-7b99-4f68-a60c-b72ae424c655 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:18.219931 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90239a-7b99-4f68-a60c-b72ae424c655 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:18.220614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90239a-7b99-4f68-a60c-b72ae424c655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:18.234394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90239a-7b99-4f68-a60c-b72ae424c655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:18.236386 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90239a-7b99-4f68-a60c-b72ae424c655 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:18.254482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 987/1964] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:42:20.202005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dec56-d322-410f-a3f1-123fa1026756 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:20.202662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dec56-d322-410f-a3f1-123fa1026756 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:20.202662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dec56-d322-410f-a3f1-123fa1026756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:20.235062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-771dec56-d322-410f-a3f1-123fa1026756 None None] MFA Rules not processed for user `399bb0de89ce4b1d828406683cad9672`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:20.236773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-771dec56-d322-410f-a3f1-123fa1026756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:20.322634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 978/1965] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:20.337160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90239a-7b99-4f68-a60c-b72ae424c655 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:20.341374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4d90239a-7b99-4f68-a60c-b72ae424c655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:20.426119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90239a-7b99-4f68-a60c-b72ae424c655 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:20.429988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0add512-90f7-4963-ae19-e913b68ba9ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d2a655f4d6f45039d641133e3b56592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:20.430927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0add512-90f7-4963-ae19-e913b68ba9ea service glance] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:20.431581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0add512-90f7-4963-ae19-e913b68ba9ea service glance] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:20.432005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0add512-90f7-4963-ae19-e913b68ba9ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:20.434655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d0add512-90f7-4963-ae19-e913b68ba9ea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:20.497263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:20.515476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0add512-90f7-4963-ae19-e913b68ba9ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:20.517198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0add512-90f7-4963-ae19-e913b68ba9ea service glance] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:20.529960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 988/1966] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:20.690440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c10b8-6e33-49b1-955f-bc8323ecc9c1 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:20.691385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c10b8-6e33-49b1-955f-bc8323ecc9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:20.691865 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c10b8-6e33-49b1-955f-bc8323ecc9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:20.718858 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-fa2c10b8-6e33-49b1-955f-bc8323ecc9c1 None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:20.720926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fa2c10b8-6e33-49b1-955f-bc8323ecc9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:20.747613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:20.801307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 979/1967] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:42:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:20.822941 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0add512-90f7-4963-ae19-e913b68ba9ea service glance] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:20.825971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d0add512-90f7-4963-ae19-e913b68ba9ea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:20.946432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0add512-90f7-4963-ae19-e913b68ba9ea service glance] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:20.952088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f200d-ee63-43a6-92b4-aaea40afd77e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:20.952810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f200d-ee63-43a6-92b4-aaea40afd77e service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:20.953284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f200d-ee63-43a6-92b4-aaea40afd77e service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:20.953640 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f200d-ee63-43a6-92b4-aaea40afd77e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:20.965477 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f200d-ee63-43a6-92b4-aaea40afd77e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:20.967922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f200d-ee63-43a6-92b4-aaea40afd77e service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:20.979195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 989/1968] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:42:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:42:22.327027 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba173e-ca4a-464b-9c19-0ccdc540bbbc None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:22.327544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba173e-ca4a-464b-9c19-0ccdc540bbbc None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:22.327544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba173e-ca4a-464b-9c19-0ccdc540bbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:22.361581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7fba173e-ca4a-464b-9c19-0ccdc540bbbc None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:22.378186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7fba173e-ca4a-464b-9c19-0ccdc540bbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:22.414359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 980/1969] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:42:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:22.425640 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f200d-ee63-43a6-92b4-aaea40afd77e service swift] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:22.447674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f200d-ee63-43a6-92b4-aaea40afd77e service swift] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:22.463809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3a50-8926-416e-9998-a4962a4a29bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:22.465902 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3a50-8926-416e-9998-a4962a4a29bf service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:22.466177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3a50-8926-416e-9998-a4962a4a29bf service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:22.466904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3a50-8926-416e-9998-a4962a4a29bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:22.499187 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf3a50-8926-416e-9998-a4962a4a29bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:22.500806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf3a50-8926-416e-9998-a4962a4a29bf service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:22.520232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 990/1970] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:42:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:42:27.929460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8e61-c9cc-4d6c-a58c-f1e72666d244 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:27.930004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8e61-c9cc-4d6c-a58c-f1e72666d244 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:27.930960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8e61-c9cc-4d6c-a58c-f1e72666d244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:28.009909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-daad8e61-c9cc-4d6c-a58c-f1e72666d244 None None] MFA Rules not processed for user `f76d70b021f44d1391653477c65f95af`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:28.011349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-daad8e61-c9cc-4d6c-a58c-f1e72666d244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:28.194891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 981/1971] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:28.198503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3a50-8926-416e-9998-a4962a4a29bf service swift] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:28.220543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3a50-8926-416e-9998-a4962a4a29bf service swift] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:28.290031 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31e4ff-c10b-40b9-b19f-1e57abb5f4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:28.290680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31e4ff-c10b-40b9-b19f-1e57abb5f4da admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:28.290964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31e4ff-c10b-40b9-b19f-1e57abb5f4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:28.298040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31e4ff-c10b-40b9-b19f-1e57abb5f4da admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:28.299046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31e4ff-c10b-40b9-b19f-1e57abb5f4da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:28.305884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31e4ff-c10b-40b9-b19f-1e57abb5f4da admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:28.395593 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a54757-6a30-4267-873e-22e641d5aa22 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:28.396054 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a54757-6a30-4267-873e-22e641d5aa22 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:28.396344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a54757-6a30-4267-873e-22e641d5aa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:28.440753 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 991/1972] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:42:28 2022] POST /identity/v3/projects => generated 371 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:28.457670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31e4ff-c10b-40b9-b19f-1e57abb5f4da admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:28.489817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-82a54757-6a30-4267-873e-22e641d5aa22 None None] MFA Rules not processed for user `1f624c36a3574b61bd17d5d73af71916`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:28.492209 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-82a54757-6a30-4267-873e-22e641d5aa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:28.574178 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31e4ff-c10b-40b9-b19f-1e57abb5f4da admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:28.579900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aa2bd-f886-4af0-b752-eb845baac163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:28.580577 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aa2bd-f886-4af0-b752-eb845baac163 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:28.580754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aa2bd-f886-4af0-b752-eb845baac163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:28.581728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4aa2bd-f886-4af0-b752-eb845baac163 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:28.585424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4aa2bd-f886-4af0-b752-eb845baac163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:28.587771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4aa2bd-f886-4af0-b752-eb845baac163 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:28.617938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 992/1973] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:42:28 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:42:28.625601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aa2bd-f886-4af0-b752-eb845baac163 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:28.653456 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 982/1974] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:42:28.656309 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4aa2bd-f886-4af0-b752-eb845baac163 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:28.665153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d53e4-be6c-4933-a073-7b27966de1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:28.666087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d53e4-be6c-4933-a073-7b27966de1ae admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:28.667246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d53e4-be6c-4933-a073-7b27966de1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:28.667694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d53e4-be6c-4933-a073-7b27966de1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:28.668599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d53e4-be6c-4933-a073-7b27966de1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:28.670558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d53e4-be6c-4933-a073-7b27966de1ae admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:28.684832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 993/1975] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:42:28.688827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a54757-6a30-4267-873e-22e641d5aa22 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:28.707802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a54757-6a30-4267-873e-22e641d5aa22 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:28.751831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806d446-46e5-46b9-88db-3acccd143633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:28.755809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806d446-46e5-46b9-88db-3acccd143633 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:28.756728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806d446-46e5-46b9-88db-3acccd143633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:28.760297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806d446-46e5-46b9-88db-3acccd143633 admin admin] PATH_INFO: `/v3/projects/36417a0eb5b448f49f22434306fc0548/users/a94981db1de14312ab19054745731580/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:28.821011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806d446-46e5-46b9-88db-3acccd143633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36417a0eb5b448f49f22434306fc0548, user_id=a94981db1de14312ab19054745731580, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:28.826957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806d446-46e5-46b9-88db-3acccd143633 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:28.864880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d53e4-be6c-4933-a073-7b27966de1ae admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:28.871914 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 983/1976] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:42:28 2022] PUT /identity/v3/projects/36417a0eb5b448f49f22434306fc0548/users/a94981db1de14312ab19054745731580/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:28.886343 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d53e4-be6c-4933-a073-7b27966de1ae admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:28.931580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:29.032221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd9c96-d9fa-49e8-a455-6564d2a85759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:29.033019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd9c96-d9fa-49e8-a455-6564d2a85759 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:29.033411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd9c96-d9fa-49e8-a455-6564d2a85759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:29.033797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd9c96-d9fa-49e8-a455-6564d2a85759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:29.034549 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd9c96-d9fa-49e8-a455-6564d2a85759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:29.036265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd9c96-d9fa-49e8-a455-6564d2a85759 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:29.071972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 994/1977] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:42:29.077106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806d446-46e5-46b9-88db-3acccd143633 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:29.095498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806d446-46e5-46b9-88db-3acccd143633 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:29.111205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f30f6-c26f-43ce-8b2a-faaecc773591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:29.111909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f30f6-c26f-43ce-8b2a-faaecc773591 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:29.112186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f30f6-c26f-43ce-8b2a-faaecc773591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:29.112453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f30f6-c26f-43ce-8b2a-faaecc773591 admin admin] PATH_INFO: `/v3/projects/36417a0eb5b448f49f22434306fc0548/users/a94981db1de14312ab19054745731580/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:29.114517 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f30f6-c26f-43ce-8b2a-faaecc773591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36417a0eb5b448f49f22434306fc0548, user_id=a94981db1de14312ab19054745731580, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:29.124874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f30f6-c26f-43ce-8b2a-faaecc773591 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:29.162649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 984/1978] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:42:29 2022] PUT /identity/v3/projects/36417a0eb5b448f49f22434306fc0548/users/a94981db1de14312ab19054745731580/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:42:30.278924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215ef8-e7e4-4817-9428-d9358937caaf None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:30.279418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215ef8-e7e4-4817-9428-d9358937caaf None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:30.279510 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215ef8-e7e4-4817-9428-d9358937caaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:30.326319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-0b215ef8-e7e4-4817-9428-d9358937caaf None None] MFA Rules not processed for user `916699a212eb4d8aa945f299ac0d4c96`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:30.327562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0b215ef8-e7e4-4817-9428-d9358937caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:30.510682 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 995/1979] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:39.021715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3adaa4-7d23-47a1-97a0-004020046ea5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:39.022268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3adaa4-7d23-47a1-97a0-004020046ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:39.022356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3adaa4-7d23-47a1-97a0-004020046ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:39.071731 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-8e3adaa4-7d23-47a1-97a0-004020046ea5 None None] MFA Rules not processed for user `a94981db1de14312ab19054745731580`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:39.075323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8e3adaa4-7d23-47a1-97a0-004020046ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:39.099875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:39.234669 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 985/1980] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:40.490305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=45 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:41.587465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b215ef8-e7e4-4817-9428-d9358937caaf None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:41.589914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0b215ef8-e7e4-4817-9428-d9358937caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:41.743889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b215ef8-e7e4-4817-9428-d9358937caaf None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:41.759423 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875906f-b617-40a2-972c-f3890d11e098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:41.760468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875906f-b617-40a2-972c-f3890d11e098 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:41.760948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875906f-b617-40a2-972c-f3890d11e098 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:41.761506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875906f-b617-40a2-972c-f3890d11e098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:41.846846 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c3ccd-c228-4f99-b33c-acdbf43f26bf None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:41.847358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c3ccd-c228-4f99-b33c-acdbf43f26bf None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:41.847567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c3ccd-c228-4f99-b33c-acdbf43f26bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:41.888531 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-5f5c3ccd-c228-4f99-b33c-acdbf43f26bf None None] MFA Rules not processed for user `b9fed14900304dc99c4c1d5fdfe204af`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:41.896546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5f5c3ccd-c228-4f99-b33c-acdbf43f26bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:41.900726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875906f-b617-40a2-972c-f3890d11e098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:41.902473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875906f-b617-40a2-972c-f3890d11e098 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:41.939034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 996/1981] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:42:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:42:42.101825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 986/1982] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:46.648434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22970315-8209-4e3a-b201-72807910e725 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:46.649843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22970315-8209-4e3a-b201-72807910e725 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:46.650338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22970315-8209-4e3a-b201-72807910e725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:46.675540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-22970315-8209-4e3a-b201-72807910e725 None None] MFA Rules not processed for user `f76d70b021f44d1391653477c65f95af`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:46.677988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-22970315-8209-4e3a-b201-72807910e725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:46.695271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 997/1983] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:42:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:47.687724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab5843-889b-45ad-ab27-71ae37ac817f None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:47.688256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab5843-889b-45ad-ab27-71ae37ac817f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:47.688480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab5843-889b-45ad-ab27-71ae37ac817f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:47.723435 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7cab5843-889b-45ad-ab27-71ae37ac817f None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:42:47.728582 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7cab5843-889b-45ad-ab27-71ae37ac817f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:42:47.854006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 987/1984] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:42:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:42:47.904603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22970315-8209-4e3a-b201-72807910e725 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:42:48.008702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22970315-8209-4e3a-b201-72807910e725 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:42:48.016061 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba1764-cbee-40da-8afa-a7e75ee58569 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:42:48.017119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1764-cbee-40da-8afa-a7e75ee58569 service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:42:48.018155 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1764-cbee-40da-8afa-a7e75ee58569 service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:42:48.018644 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1764-cbee-40da-8afa-a7e75ee58569 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:42:48.043504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba1764-cbee-40da-8afa-a7e75ee58569 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:42:48.045759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba1764-cbee-40da-8afa-a7e75ee58569 service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:42:48.057596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 998/1985] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:42:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:42:57.843796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=75 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:42:58.052503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=91 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:43:00.241028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbce0b3-42f9-4e53-9699-917b1c1b449f None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:00.241548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbce0b3-42f9-4e53-9699-917b1c1b449f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:00.241650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbce0b3-42f9-4e53-9699-917b1c1b449f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:00.266519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-fcbce0b3-42f9-4e53-9699-917b1c1b449f None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:00.267791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fcbce0b3-42f9-4e53-9699-917b1c1b449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:00.275408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 988/1986] 10.222.0.17 () {58 vars in 967 bytes} [Tue Oct 18 11:43:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:43:00.299895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba1764-cbee-40da-8afa-a7e75ee58569 service swift] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:00.311133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba1764-cbee-40da-8afa-a7e75ee58569 service swift] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:00.316832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0af63-9992-49b7-b755-9811926503a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:00.317497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0af63-9992-49b7-b755-9811926503a3 service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:00.318031 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0af63-9992-49b7-b755-9811926503a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:00.318342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0af63-9992-49b7-b755-9811926503a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:00.328748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0af63-9992-49b7-b755-9811926503a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:00.330502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0af63-9992-49b7-b755-9811926503a3 service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:00.339561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 999/1987] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:43:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:43:02.621008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbce0b3-42f9-4e53-9699-917b1c1b449f None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:02.622612 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fcbce0b3-42f9-4e53-9699-917b1c1b449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:02.717360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbce0b3-42f9-4e53-9699-917b1c1b449f None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:02.722556 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:02.723285 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:02.723629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:02.723964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] PATH_INFO: `/v3/users/fcb7dccab5ec46348c87ad4889120109` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:02.725904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcb7dccab5ec46348c87ad4889120109)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:02.727964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:02.806200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcb7dccab5ec46348c87ad4889120109'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:02.814696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcb7dccab5ec46348c87ad4889120109'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:02.815446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcb7dccab5ec46348c87ad4889120109'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:02.828675 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcb7dccab5ec46348c87ad4889120109'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:02.842548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-39770177-aef8-4f11-9683-066ab1c801e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcb7dccab5ec46348c87ad4889120109'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:02.849508 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 989/1988] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:43:02 2022] DELETE /identity/v3/users/fcb7dccab5ec46348c87ad4889120109 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:43:03.033605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0af63-9992-49b7-b755-9811926503a3 service swift] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:03.045374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0af63-9992-49b7-b755-9811926503a3 service swift] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:03.105501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:03.106144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.106427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.106701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] PATH_INFO: `/v3/projects/047b4fe1eab744d9b871985c30e76b0d` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.109603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=047b4fe1eab744d9b871985c30e76b0d)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:03.111541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:03.166413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '047b4fe1eab744d9b871985c30e76b0d', 'request_id': 'req-63efa7f8-3cf2-447d-ae82-044d01de03a6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:03.173241 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '047b4fe1eab744d9b871985c30e76b0d', 'request_id': 'req-63efa7f8-3cf2-447d-ae82-044d01de03a6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:03.184114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '047b4fe1eab744d9b871985c30e76b0d', 'request_id': 'req-63efa7f8-3cf2-447d-ae82-044d01de03a6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:03.197890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] The token cache is being invalidate because project 047b4fe1eab744d9b871985c30e76b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:43:03.198185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:03.198906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-63efa7f8-3cf2-447d-ae82-044d01de03a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:03.200395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1000/1989] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:43:03 2022] DELETE /identity/v3/projects/047b4fe1eab744d9b871985c30e76b0d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:43:03.205428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16340d55-5830-4a5d-8981-de090138ab7f None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.205716 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16340d55-5830-4a5d-8981-de090138ab7f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.205989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16340d55-5830-4a5d-8981-de090138ab7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.226062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-16340d55-5830-4a5d-8981-de090138ab7f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:03.227290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-16340d55-5830-4a5d-8981-de090138ab7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:03.291065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 990/1990] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:43:03.298683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecbaf4-a5ce-4d41-bbfb-4f8db2beddc4 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.298992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecbaf4-a5ce-4d41-bbfb-4f8db2beddc4 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.299274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecbaf4-a5ce-4d41-bbfb-4f8db2beddc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.319494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-c6ecbaf4-a5ce-4d41-bbfb-4f8db2beddc4 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:03.320566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c6ecbaf4-a5ce-4d41-bbfb-4f8db2beddc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:03.329605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1001/1991] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:43:03.334539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16340d55-5830-4a5d-8981-de090138ab7f None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:03.344900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16340d55-5830-4a5d-8981-de090138ab7f None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:03.350062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bca99f-41c7-4376-bd04-5846c3cb6967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:03.350667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca99f-41c7-4376-bd04-5846c3cb6967 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.350951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca99f-41c7-4376-bd04-5846c3cb6967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.351215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca99f-41c7-4376-bd04-5846c3cb6967 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.351855 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca99f-41c7-4376-bd04-5846c3cb6967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:43:03.352159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca99f-41c7-4376-bd04-5846c3cb6967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:03.353783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca99f-41c7-4376-bd04-5846c3cb6967 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:03.362329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:43:03.366455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 991/1992] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:43:03.370711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecbaf4-a5ce-4d41-bbfb-4f8db2beddc4 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:03.380079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecbaf4-a5ce-4d41-bbfb-4f8db2beddc4 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:03.384754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821a83b-cd62-47a5-a6bd-8b43808244ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:03.385392 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821a83b-cd62-47a5-a6bd-8b43808244ae admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.385674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821a83b-cd62-47a5-a6bd-8b43808244ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.385950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821a83b-cd62-47a5-a6bd-8b43808244ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.386639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821a83b-cd62-47a5-a6bd-8b43808244ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:03.388590 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821a83b-cd62-47a5-a6bd-8b43808244ae admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:03.417568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1002/1993] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:43:03 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:43:03.421998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bca99f-41c7-4376-bd04-5846c3cb6967 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:03.433162 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bca99f-41c7-4376-bd04-5846c3cb6967 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:03.490026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049c219-4e78-45ba-96ee-2dd36ce3ae61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:03.490991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049c219-4e78-45ba-96ee-2dd36ce3ae61 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.491264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049c219-4e78-45ba-96ee-2dd36ce3ae61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.491533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049c219-4e78-45ba-96ee-2dd36ce3ae61 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.492192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049c219-4e78-45ba-96ee-2dd36ce3ae61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:03.494185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049c219-4e78-45ba-96ee-2dd36ce3ae61 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:03.518046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 992/1994] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:43:03 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:43:03.519960 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821a83b-cd62-47a5-a6bd-8b43808244ae admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:03.531515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821a83b-cd62-47a5-a6bd-8b43808244ae admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:03.536220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d580165-5bb4-464e-b71c-04d89496dded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:03.536822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580165-5bb4-464e-b71c-04d89496dded admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.537142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580165-5bb4-464e-b71c-04d89496dded admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.538002 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580165-5bb4-464e-b71c-04d89496dded admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.538648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d580165-5bb4-464e-b71c-04d89496dded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:03.540236 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d580165-5bb4-464e-b71c-04d89496dded admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:03.551441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1003/1995] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:43:03.555546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049c219-4e78-45ba-96ee-2dd36ce3ae61 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:03.565716 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049c219-4e78-45ba-96ee-2dd36ce3ae61 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:03.570866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54274b60-dc7f-48ff-853b-28e604adb56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:03.571571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54274b60-dc7f-48ff-853b-28e604adb56d admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.571950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54274b60-dc7f-48ff-853b-28e604adb56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.572222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54274b60-dc7f-48ff-853b-28e604adb56d admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.573022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54274b60-dc7f-48ff-853b-28e604adb56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:03.574992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54274b60-dc7f-48ff-853b-28e604adb56d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:03.587204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 993/1996] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:43:03.591265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d580165-5bb4-464e-b71c-04d89496dded admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:03.601764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d580165-5bb4-464e-b71c-04d89496dded admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:03.606471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19738f8-f7a6-4a5d-8ff1-ed86c2fdae88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:03.607112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19738f8-f7a6-4a5d-8ff1-ed86c2fdae88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.607385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19738f8-f7a6-4a5d-8ff1-ed86c2fdae88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.607664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19738f8-f7a6-4a5d-8ff1-ed86c2fdae88 admin admin] PATH_INFO: `/v3/projects/b3c795bc9e2549cc8c3bd34a998cfda5/users/6a130451f4b840ad87aa53671e8ea6a4/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.623335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19738f8-f7a6-4a5d-8ff1-ed86c2fdae88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c795bc9e2549cc8c3bd34a998cfda5, user_id=6a130451f4b840ad87aa53671e8ea6a4, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:03.626216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19738f8-f7a6-4a5d-8ff1-ed86c2fdae88 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:03.639158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1004/1997] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:43:03 2022] PUT /identity/v3/projects/b3c795bc9e2549cc8c3bd34a998cfda5/users/6a130451f4b840ad87aa53671e8ea6a4/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:43:03.650897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbe405-3624-4fe7-afa0-fc87807e71b0 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.651399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbe405-3624-4fe7-afa0-fc87807e71b0 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.651529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbe405-3624-4fe7-afa0-fc87807e71b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.681455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7adbe405-3624-4fe7-afa0-fc87807e71b0 None None] MFA Rules not processed for user `6a130451f4b840ad87aa53671e8ea6a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:03.682808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7adbe405-3624-4fe7-afa0-fc87807e71b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:03.768132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 994/1998] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:43:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:43:03.780574 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19738f8-f7a6-4a5d-8ff1-ed86c2fdae88 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:03.782471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d19738f8-f7a6-4a5d-8ff1-ed86c2fdae88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:03.807124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:43:03.880308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19738f8-f7a6-4a5d-8ff1-ed86c2fdae88 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:03.890029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43183d37-6f8f-485f-b479-2178ce10449c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:03.891053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43183d37-6f8f-485f-b479-2178ce10449c service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:03.891589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43183d37-6f8f-485f-b479-2178ce10449c service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:03.894265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43183d37-6f8f-485f-b479-2178ce10449c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:03.918014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43183d37-6f8f-485f-b479-2178ce10449c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:03.919574 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43183d37-6f8f-485f-b479-2178ce10449c service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:03.931533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1005/1999] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:43:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:43:13.764331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:43:13.926322 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=20 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:43:23.545557 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbe405-3624-4fe7-afa0-fc87807e71b0 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:23.611242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbe405-3624-4fe7-afa0-fc87807e71b0 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:23.616107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:23.616741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:23.617043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:23.617314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] PATH_INFO: `/v3/users/6a130451f4b840ad87aa53671e8ea6a4` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:23.618464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a130451f4b840ad87aa53671e8ea6a4)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:23.620379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:23.668858 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a130451f4b840ad87aa53671e8ea6a4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:23.676116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a130451f4b840ad87aa53671e8ea6a4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:23.676996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a130451f4b840ad87aa53671e8ea6a4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:23.692970 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a130451f4b840ad87aa53671e8ea6a4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:23.702813 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a130451f4b840ad87aa53671e8ea6a4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:23.709443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 995/2000] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:43:23 2022] DELETE /identity/v3/users/6a130451f4b840ad87aa53671e8ea6a4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:43:23.714316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310b20-09f9-41e5-94c9-3e561f1651ad None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:23.714837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310b20-09f9-41e5-94c9-3e561f1651ad None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:23.714837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94310b20-09f9-41e5-94c9-3e561f1651ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:23.736961 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-94310b20-09f9-41e5-94c9-3e561f1651ad None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:23.738161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-94310b20-09f9-41e5-94c9-3e561f1651ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:23.799468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1006/2001] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:43:23.810731 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:23.812542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:23.872510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdacc4-f251-4259-b64f-064ea995edb6 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:23.876254 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c5500-e94c-4483-a274-e4c393e467e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:23.876898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c5500-e94c-4483-a274-e4c393e467e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:23.877179 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c5500-e94c-4483-a274-e4c393e467e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:23.877453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c5500-e94c-4483-a274-e4c393e467e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:23.886750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c5500-e94c-4483-a274-e4c393e467e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:23.888189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c5500-e94c-4483-a274-e4c393e467e1 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:23.898374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 996/2002] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:43:24.166780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94310b20-09f9-41e5-94c9-3e561f1651ad None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:24.168601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-94310b20-09f9-41e5-94c9-3e561f1651ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:24.188420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94310b20-09f9-41e5-94c9-3e561f1651ad None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:24.193630 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:24.194329 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:24.194530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:24.194799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] PATH_INFO: `/v3/projects/b3c795bc9e2549cc8c3bd34a998cfda5` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:24.195931 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c795bc9e2549cc8c3bd34a998cfda5)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:24.197882 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:24.244147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c795bc9e2549cc8c3bd34a998cfda5', 'request_id': 'req-7ca55853-09c4-4b50-bdd0-2f909b92da86'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:24.249785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c795bc9e2549cc8c3bd34a998cfda5', 'request_id': 'req-7ca55853-09c4-4b50-bdd0-2f909b92da86'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:24.257416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c795bc9e2549cc8c3bd34a998cfda5', 'request_id': 'req-7ca55853-09c4-4b50-bdd0-2f909b92da86'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:24.268988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] The token cache is being invalidate because project b3c795bc9e2549cc8c3bd34a998cfda5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:43:24.269274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:24.269894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7ca55853-09c4-4b50-bdd0-2f909b92da86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:43:24.272115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1007/2003] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:43:24 2022] DELETE /identity/v3/projects/b3c795bc9e2549cc8c3bd34a998cfda5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:43:27.964075 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a68d11-2b4c-46b6-93c4-ea372d42e09c None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:27.964621 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a68d11-2b4c-46b6-93c4-ea372d42e09c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:27.964721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a68d11-2b4c-46b6-93c4-ea372d42e09c None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:27.966935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 997/2004] 10.222.0.17 () {54 vars in 961 bytes} [Tue Oct 18 11:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:43:27.972305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ae978-31fa-48c5-8bdd-b77c4b87ad89 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:27.972812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ae978-31fa-48c5-8bdd-b77c4b87ad89 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:27.973104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ae978-31fa-48c5-8bdd-b77c4b87ad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:27.996387 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-7c9ae978-31fa-48c5-8bdd-b77c4b87ad89 None None] MFA Rules not processed for user `99ec4a4276f345c2998723e554ab9608`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:27.997674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7c9ae978-31fa-48c5-8bdd-b77c4b87ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:28.103289 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1008/2005] 10.222.0.17 () {58 vars in 1058 bytes} [Tue Oct 18 11:43:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:43:28.109913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a68d11-2b4c-46b6-93c4-ea372d42e09c None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:28.159506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a68d11-2b4c-46b6-93c4-ea372d42e09c None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:28.167362 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17260b-19f6-4bbe-acab-411d9fb8b61a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:28.168103 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17260b-19f6-4bbe-acab-411d9fb8b61a service placement] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:28.168661 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17260b-19f6-4bbe-acab-411d9fb8b61a service placement] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:28.192646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17260b-19f6-4bbe-acab-411d9fb8b61a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:28.198933 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5a17260b-19f6-4bbe-acab-411d9fb8b61a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:28.236115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17260b-19f6-4bbe-acab-411d9fb8b61a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:28.238605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17260b-19f6-4bbe-acab-411d9fb8b61a service placement] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:28.253343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 998/2006] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:43:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:43:28.398898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ae978-31fa-48c5-8bdd-b77c4b87ad89 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:28.400288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7c9ae978-31fa-48c5-8bdd-b77c4b87ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:28.420825 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:43:28.457887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ae978-31fa-48c5-8bdd-b77c4b87ad89 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:28.461963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca55ca-2a3e-441d-bb65-78d340c32c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:28.462562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca55ca-2a3e-441d-bb65-78d340c32c9c service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:28.462835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca55ca-2a3e-441d-bb65-78d340c32c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:28.463101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca55ca-2a3e-441d-bb65-78d340c32c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:28.464797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d6ca55ca-2a3e-441d-bb65-78d340c32c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:28.524526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca55ca-2a3e-441d-bb65-78d340c32c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:28.526001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca55ca-2a3e-441d-bb65-78d340c32c9c service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:28.535252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1009/2007] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:43:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:43:33.383007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c3f4a-2940-43ed-a300-932b13cecb3d None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:33.383485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c3f4a-2940-43ed-a300-932b13cecb3d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:33.383580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c3f4a-2940-43ed-a300-932b13cecb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:33.408501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-422c3f4a-2940-43ed-a300-932b13cecb3d None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:33.409784 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-422c3f4a-2940-43ed-a300-932b13cecb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:33.414714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:43:33.471321 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 999/2008] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:43:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:43:33.482614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca55ca-2a3e-441d-bb65-78d340c32c9c service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:33.484783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d6ca55ca-2a3e-441d-bb65-78d340c32c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:33.564813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca55ca-2a3e-441d-bb65-78d340c32c9c service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:33.570098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b1504-cb3e-42b6-bacb-8ed94c7887ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:33.570707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b1504-cb3e-42b6-bacb-8ed94c7887ea service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:33.570983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b1504-cb3e-42b6-bacb-8ed94c7887ea service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:33.571252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b1504-cb3e-42b6-bacb-8ed94c7887ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:33.580352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b1504-cb3e-42b6-bacb-8ed94c7887ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:33.581967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b1504-cb3e-42b6-bacb-8ed94c7887ea service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:33.594847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1010/2009] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:43:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:43:37.499513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c3f4a-2940-43ed-a300-932b13cecb3d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:37.501073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-422c3f4a-2940-43ed-a300-932b13cecb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:37.515098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c3f4a-2940-43ed-a300-932b13cecb3d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:37.519114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f521a4-2c66-4dc7-b450-3f0196ae8c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:37.519821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f521a4-2c66-4dc7-b450-3f0196ae8c29 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:37.520167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f521a4-2c66-4dc7-b450-3f0196ae8c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:37.520450 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f521a4-2c66-4dc7-b450-3f0196ae8c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:37.522442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-73f521a4-2c66-4dc7-b450-3f0196ae8c29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:37.631697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f521a4-2c66-4dc7-b450-3f0196ae8c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:37.633273 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f521a4-2c66-4dc7-b450-3f0196ae8c29 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:37.644421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1000/2010] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:43:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:43:43.590555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=73 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:43:47.369477 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a9dc0-01c5-412f-a48f-8579e71b95d9 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:47.370445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a9dc0-01c5-412f-a48f-8579e71b95d9 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:47.370805 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a9dc0-01c5-412f-a48f-8579e71b95d9 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:47.373395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1011/2011] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:43:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:43:47.379618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdc7e5-7fc0-45be-8ba1-c622f19fc729 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:47.380169 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdc7e5-7fc0-45be-8ba1-c622f19fc729 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:47.380359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdc7e5-7fc0-45be-8ba1-c622f19fc729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:47.407835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d0fdc7e5-7fc0-45be-8ba1-c622f19fc729 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:47.409166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d0fdc7e5-7fc0-45be-8ba1-c622f19fc729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:47.496532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1001/2012] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:43:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:43:47.508520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a9dc0-01c5-412f-a48f-8579e71b95d9 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:47.510193 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-530a9dc0-01c5-412f-a48f-8579e71b95d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:47.572311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a9dc0-01c5-412f-a48f-8579e71b95d9 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:47.575896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a6361-48f2-4ebf-8942-cca2982aa384 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d2a655f4d6f45039d641133e3b56592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:47.576500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a6361-48f2-4ebf-8942-cca2982aa384 service glance] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:47.576777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a6361-48f2-4ebf-8942-cca2982aa384 service glance] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:47.577077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a6361-48f2-4ebf-8942-cca2982aa384 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:47.588168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a6361-48f2-4ebf-8942-cca2982aa384 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:47.589625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a6361-48f2-4ebf-8942-cca2982aa384 service glance] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:47.600255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1012/2013] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:43:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:43:47.668145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ced1-83f3-476d-8e3a-68b8ecbdba01 None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:47.668425 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ced1-83f3-476d-8e3a-68b8ecbdba01 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:47.668698 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ced1-83f3-476d-8e3a-68b8ecbdba01 None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:47.670076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1002/2014] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:43:47.674680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925a51f-d2d6-48b0-a6cf-a2f63ace99ae None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:47.674966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925a51f-d2d6-48b0-a6cf-a2f63ace99ae None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:47.675238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925a51f-d2d6-48b0-a6cf-a2f63ace99ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:47.695646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-3925a51f-d2d6-48b0-a6cf-a2f63ace99ae None None] MFA Rules not processed for user `d301f3d9aeb644a38c31dbdbef2a48c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:47.696953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3925a51f-d2d6-48b0-a6cf-a2f63ace99ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:47.703623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1013/2015] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:43:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:43:47.715666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1ced1-83f3-476d-8e3a-68b8ecbdba01 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:47.727586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1ced1-83f3-476d-8e3a-68b8ecbdba01 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:47.731112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c74f48-3488-47c2-8117-ab0172040f76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:47.731705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c74f48-3488-47c2-8117-ab0172040f76 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:47.731978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c74f48-3488-47c2-8117-ab0172040f76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:47.732245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c74f48-3488-47c2-8117-ab0172040f76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:47.739406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c74f48-3488-47c2-8117-ab0172040f76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:47.741410 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c74f48-3488-47c2-8117-ab0172040f76 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:47.748954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1003/2016] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:43:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:43:47.831796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09959418-31af-4e15-99f8-adb2095180f1 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:47.832086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09959418-31af-4e15-99f8-adb2095180f1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:47.832359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09959418-31af-4e15-99f8-adb2095180f1 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:47.833745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1014/2017] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:43:47.838922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7da6a-f040-489c-95c2-dddeadc7b155 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:47.839202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7da6a-f040-489c-95c2-dddeadc7b155 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:47.839470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7da6a-f040-489c-95c2-dddeadc7b155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:47.859255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4bf7da6a-f040-489c-95c2-dddeadc7b155 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:47.860438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4bf7da6a-f040-489c-95c2-dddeadc7b155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:47.870944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1004/2018] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:43:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:43:47.883982 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09959418-31af-4e15-99f8-adb2095180f1 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:47.896567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09959418-31af-4e15-99f8-adb2095180f1 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:47.900142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7a74c-6f0e-4286-af00-5f9ad629cfbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:47.900762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7a74c-6f0e-4286-af00-5f9ad629cfbf service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:47.901065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7a74c-6f0e-4286-af00-5f9ad629cfbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:47.901334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7a74c-6f0e-4286-af00-5f9ad629cfbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:47.912269 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7a74c-6f0e-4286-af00-5f9ad629cfbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:47.913757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7a74c-6f0e-4286-af00-5f9ad629cfbf service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:47.924693 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1015/2019] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:43:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:43:51.202594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8a64c-1cf0-4090-a2d1-f3b92d8f83a0 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:51.203042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8a64c-1cf0-4090-a2d1-f3b92d8f83a0 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:51.203169 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8a64c-1cf0-4090-a2d1-f3b92d8f83a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:51.223556 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-55b8a64c-1cf0-4090-a2d1-f3b92d8f83a0 None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:43:51.225298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-55b8a64c-1cf0-4090-a2d1-f3b92d8f83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:43:51.232774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1005/2020] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:43:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:43:51.245795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7a74c-6f0e-4286-af00-5f9ad629cfbf service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:43:51.257515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7a74c-6f0e-4286-af00-5f9ad629cfbf service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:43:51.262491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28248442-33eb-409b-a0e8-e1fd752f7eff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:43:51.263087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28248442-33eb-409b-a0e8-e1fd752f7eff service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:43:51.263364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28248442-33eb-409b-a0e8-e1fd752f7eff service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:43:51.263634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28248442-33eb-409b-a0e8-e1fd752f7eff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:43:51.272484 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28248442-33eb-409b-a0e8-e1fd752f7eff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:43:51.274116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28248442-33eb-409b-a0e8-e1fd752f7eff service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:43:51.281972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1016/2021] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:43:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:44:01.218714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=94 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:44:01.278752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=41 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:44:14.935975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064f525-9345-4477-af6d-b6ce7f866eb0 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:44:14.936773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064f525-9345-4477-af6d-b6ce7f866eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:44:14.937087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064f525-9345-4477-af6d-b6ce7f866eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:44:14.961457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-6064f525-9345-4477-af6d-b6ce7f866eb0 None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:44:14.962652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6064f525-9345-4477-af6d-b6ce7f866eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:44:14.969891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1006/2022] 10.222.0.17 () {58 vars in 967 bytes} [Tue Oct 18 11:44:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:44:14.992391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28248442-33eb-409b-a0e8-e1fd752f7eff service swift] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:44:15.002749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28248442-33eb-409b-a0e8-e1fd752f7eff service swift] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:44:15.007759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6244e6-ac00-4f91-b1a1-4c8b603c86c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:44:15.008355 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6244e6-ac00-4f91-b1a1-4c8b603c86c7 service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:44:15.008659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6244e6-ac00-4f91-b1a1-4c8b603c86c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:44:15.008970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6244e6-ac00-4f91-b1a1-4c8b603c86c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:44:15.016803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6244e6-ac00-4f91-b1a1-4c8b603c86c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:44:15.018401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6244e6-ac00-4f91-b1a1-4c8b603c86c7 service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:44:15.026319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1017/2023] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:44:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:44:24.956258 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:44:25.023074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=6 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:44:31.165905 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28a6a3-48ef-413f-84ba-7c0bc6059418 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:44:31.166473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28a6a3-48ef-413f-84ba-7c0bc6059418 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:44:31.166473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28a6a3-48ef-413f-84ba-7c0bc6059418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:44:31.188725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ee28a6a3-48ef-413f-84ba-7c0bc6059418 None None] MFA Rules not processed for user `916699a212eb4d8aa945f299ac0d4c96`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:44:31.189989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ee28a6a3-48ef-413f-84ba-7c0bc6059418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:44:31.246989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1007/2024] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:44:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:44:34.603554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c534b-2a7b-49f9-bf78-4d0dd7555598 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:44:34.604130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c534b-2a7b-49f9-bf78-4d0dd7555598 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:44:34.604130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c534b-2a7b-49f9-bf78-4d0dd7555598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:44:34.623873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-342c534b-2a7b-49f9-bf78-4d0dd7555598 None None] MFA Rules not processed for user `1f624c36a3574b61bd17d5d73af71916`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:44:34.625101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-342c534b-2a7b-49f9-bf78-4d0dd7555598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:44:34.685216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1018/2025] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:44:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:44:41.239725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=25 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:44:44.675394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=25 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:01.408885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb1509-80a0-4d80-92e9-188c8ccab3fc None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:01.409818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb1509-80a0-4d80-92e9-188c8ccab3fc None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:01.410080 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb1509-80a0-4d80-92e9-188c8ccab3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:01.429573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a6eb1509-80a0-4d80-92e9-188c8ccab3fc None None] MFA Rules not processed for user `bddd3d2bb8f647f2ba616141eb16622e`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:01.430770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a6eb1509-80a0-4d80-92e9-188c8ccab3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:01.485571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1008/2026] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:07.324989 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d51e14-14a7-48cb-a690-b4d202993596 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:07.325887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d51e14-14a7-48cb-a690-b4d202993596 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:07.326164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d51e14-14a7-48cb-a690-b4d202993596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:07.345938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-74d51e14-14a7-48cb-a690-b4d202993596 None None] MFA Rules not processed for user `72d748ebce2742058757304095d1a641`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:07.347395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-74d51e14-14a7-48cb-a690-b4d202993596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:07.406303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1019/2027] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:11.478665 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=25 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:17.399452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=25 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:19.930756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb1509-80a0-4d80-92e9-188c8ccab3fc None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:19.940572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb1509-80a0-4d80-92e9-188c8ccab3fc None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:19.944882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef9f93-f330-44b1-8f8a-af89239b38fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:19.945512 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef9f93-f330-44b1-8f8a-af89239b38fd service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:19.945787 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef9f93-f330-44b1-8f8a-af89239b38fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:19.946053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef9f93-f330-44b1-8f8a-af89239b38fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:19.947758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ceef9f93-f330-44b1-8f8a-af89239b38fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:19.961653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef9f93-f330-44b1-8f8a-af89239b38fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:19.963386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef9f93-f330-44b1-8f8a-af89239b38fd service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:19.974110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1009/2028] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:45:20.156787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d51e14-14a7-48cb-a690-b4d202993596 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:20.166707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d51e14-14a7-48cb-a690-b4d202993596 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:20.170624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278eb2c-be90-4c9a-a6ca-9d5ae832009b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:20.171238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278eb2c-be90-4c9a-a6ca-9d5ae832009b service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:20.171544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278eb2c-be90-4c9a-a6ca-9d5ae832009b service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:20.171819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278eb2c-be90-4c9a-a6ca-9d5ae832009b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:20.173902 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4278eb2c-be90-4c9a-a6ca-9d5ae832009b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:20.188835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278eb2c-be90-4c9a-a6ca-9d5ae832009b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:20.190334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278eb2c-be90-4c9a-a6ca-9d5ae832009b service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:20.202386 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1020/2029] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:45:29.968654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=6 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:30.196964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=6 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:32.583467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb4115-1a30-43a3-9b80-c075784c16bf None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:32.584003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb4115-1a30-43a3-9b80-c075784c16bf None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:32.584145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb4115-1a30-43a3-9b80-c075784c16bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:32.605208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e5cb4115-1a30-43a3-9b80-c075784c16bf None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:32.606422 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e5cb4115-1a30-43a3-9b80-c075784c16bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:32.614315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1010/2030] 10.222.0.17 () {58 vars in 967 bytes} [Tue Oct 18 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:32.671777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278eb2c-be90-4c9a-a6ca-9d5ae832009b service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:32.683584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278eb2c-be90-4c9a-a6ca-9d5ae832009b service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:32.688830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4e5b6-8e73-420a-84d0-952d1e86c8f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:32.689466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4e5b6-8e73-420a-84d0-952d1e86c8f8 service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:32.689742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4e5b6-8e73-420a-84d0-952d1e86c8f8 service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:32.690008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4e5b6-8e73-420a-84d0-952d1e86c8f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:32.699643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4e5b6-8e73-420a-84d0-952d1e86c8f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:32.701388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4e5b6-8e73-420a-84d0-952d1e86c8f8 service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:32.711595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1021/2031] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:45:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:45:36.053120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb4115-1a30-43a3-9b80-c075784c16bf None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:36.054610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e5cb4115-1a30-43a3-9b80-c075784c16bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:36.073880 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb4115-1a30-43a3-9b80-c075784c16bf None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:36.079921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960866e-10fe-4013-9061-169c57e94d70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:36.080799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960866e-10fe-4013-9061-169c57e94d70 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:36.082172 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960866e-10fe-4013-9061-169c57e94d70 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:36.083255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960866e-10fe-4013-9061-169c57e94d70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:36.085997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f960866e-10fe-4013-9061-169c57e94d70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:36.099708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960866e-10fe-4013-9061-169c57e94d70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:36.101547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960866e-10fe-4013-9061-169c57e94d70 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:36.111977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1011/2032] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:45:37.937555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4e5b6-8e73-420a-84d0-952d1e86c8f8 service swift] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:37.939984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-b9e4e5b6-8e73-420a-84d0-952d1e86c8f8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:38.074067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4e5b6-8e73-420a-84d0-952d1e86c8f8 service swift] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:38.082127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9b30e-5710-4576-b21d-850cc55e8e2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:38.084745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9b30e-5710-4576-b21d-850cc55e8e2a service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:38.085303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9b30e-5710-4576-b21d-850cc55e8e2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:38.085790 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9b30e-5710-4576-b21d-850cc55e8e2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:38.088119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c1d9b30e-5710-4576-b21d-850cc55e8e2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:38.105847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9b30e-5710-4576-b21d-850cc55e8e2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:38.108515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9b30e-5710-4576-b21d-850cc55e8e2a service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:38.136437 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1022/2033] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:45:40.435396 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960866e-10fe-4013-9061-169c57e94d70 service nova] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:40.436950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f960866e-10fe-4013-9061-169c57e94d70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:40.506912 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960866e-10fe-4013-9061-169c57e94d70 service nova] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:40.511899 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:40.512543 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:40.512821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:40.513125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] PATH_INFO: `/v3/users/b9fed14900304dc99c4c1d5fdfe204af` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:40.514265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9fed14900304dc99c4c1d5fdfe204af)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:40.516214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:40.563811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9fed14900304dc99c4c1d5fdfe204af'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:40.571337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9fed14900304dc99c4c1d5fdfe204af'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:40.572017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9fed14900304dc99c4c1d5fdfe204af'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:40.585247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9fed14900304dc99c4c1d5fdfe204af'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:40.596191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9fed14900304dc99c4c1d5fdfe204af'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:40.602550 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1012/2034] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:45:40 2022] DELETE /identity/v3/users/b9fed14900304dc99c4c1d5fdfe204af => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:40.780202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9b30e-5710-4576-b21d-850cc55e8e2a service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:40.791045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9b30e-5710-4576-b21d-850cc55e8e2a service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:40.849433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:40.850021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:40.850302 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:40.850578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] PATH_INFO: `/v3/projects/b63f8bd1dce742b09e413947db416e66` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:40.851701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63f8bd1dce742b09e413947db416e66)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:40.853650 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:40.899998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63f8bd1dce742b09e413947db416e66', 'request_id': 'req-adb4416b-944b-4123-9ce5-16d05b7a7f8b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:40.905592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63f8bd1dce742b09e413947db416e66', 'request_id': 'req-adb4416b-944b-4123-9ce5-16d05b7a7f8b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:40.912810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63f8bd1dce742b09e413947db416e66', 'request_id': 'req-adb4416b-944b-4123-9ce5-16d05b7a7f8b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:40.924251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] The token cache is being invalidate because project b63f8bd1dce742b09e413947db416e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:45:40.924555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:40.925206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:40.927215 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1023/2035] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:45:40 2022] DELETE /identity/v3/projects/b63f8bd1dce742b09e413947db416e66 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:40.931265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:40.932726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:40.946866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950d579-051c-4bba-9f1c-985b4b44c5c5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:41.006098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:41.006734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:41.007009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:41.007285 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] PATH_INFO: `/v3/users/399bb0de89ce4b1d828406683cad9672` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:41.008387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=399bb0de89ce4b1d828406683cad9672)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:41.010464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:41.011933 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:41.060493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '399bb0de89ce4b1d828406683cad9672'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:41.068406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '399bb0de89ce4b1d828406683cad9672'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:41.069114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '399bb0de89ce4b1d828406683cad9672'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:41.081138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '399bb0de89ce4b1d828406683cad9672'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:41.088667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '399bb0de89ce4b1d828406683cad9672'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:41.094666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1013/2036] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:45:40 2022] DELETE /identity/v3/users/399bb0de89ce4b1d828406683cad9672 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:41.271034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:41.272616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:41.287207 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4416b-944b-4123-9ce5-16d05b7a7f8b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:41.305373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:41.347285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:41.348499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:41.348773 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:41.349139 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] PATH_INFO: `/v3/projects/657b38c99c63453b929ae9d2adb05d1f` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:41.350276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=657b38c99c63453b929ae9d2adb05d1f)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:41.352175 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:41.411526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657b38c99c63453b929ae9d2adb05d1f', 'request_id': 'req-4a871be5-4c3a-43c6-b627-a9172c425029'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:41.418199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657b38c99c63453b929ae9d2adb05d1f', 'request_id': 'req-4a871be5-4c3a-43c6-b627-a9172c425029'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:41.427014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657b38c99c63453b929ae9d2adb05d1f', 'request_id': 'req-4a871be5-4c3a-43c6-b627-a9172c425029'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:41.438853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] The token cache is being invalidate because project 657b38c99c63453b929ae9d2adb05d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:45:41.439130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:41.439798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:41.442137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1024/2037] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:45:41 2022] DELETE /identity/v3/projects/657b38c99c63453b929ae9d2adb05d1f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:47.306324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:47.307785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:47.368197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707896b-33ee-4ca5-ac85-02628b4b1881 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:47.418206 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffadb86-d5f5-432b-ab29-1bec25b89407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:47.418546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffadb86-d5f5-432b-ab29-1bec25b89407 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:47.418826 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffadb86-d5f5-432b-ab29-1bec25b89407 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:47.419017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffadb86-d5f5-432b-ab29-1bec25b89407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:47.423735 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-1ffadb86-d5f5-432b-ab29-1bec25b89407 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:47.479453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:47.505344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffadb86-d5f5-432b-ab29-1bec25b89407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:47.506937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffadb86-d5f5-432b-ab29-1bec25b89407 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:47.526860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1014/2038] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:45:51.159144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:51.161149 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:51.182635 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a871be5-4c3a-43c6-b627-a9172c425029 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:51.187315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:51.187967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:51.188245 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:51.188515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] PATH_INFO: `/v3/users/1f624c36a3574b61bd17d5d73af71916` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:51.189689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f624c36a3574b61bd17d5d73af71916)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:51.191624 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:51.245088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f624c36a3574b61bd17d5d73af71916'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:51.258654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f624c36a3574b61bd17d5d73af71916'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:51.259310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f624c36a3574b61bd17d5d73af71916'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:51.267097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f624c36a3574b61bd17d5d73af71916'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:51.267845 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-50ebac00-9e84-4305-abf0-572a3ff3792c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f624c36a3574b61bd17d5d73af71916'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:51.286377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1025/2039] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:45:51 2022] DELETE /identity/v3/users/1f624c36a3574b61bd17d5d73af71916 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:51.470692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffadb86-d5f5-432b-ab29-1bec25b89407 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:51.494409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffadb86-d5f5-432b-ab29-1bec25b89407 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:51.554004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:51.554645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:51.554918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:51.555189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] PATH_INFO: `/v3/projects/cd52252f24114465af9bdf6e6fe68b76` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:51.556387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd52252f24114465af9bdf6e6fe68b76)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:51.558548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:51.608622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd52252f24114465af9bdf6e6fe68b76', 'request_id': 'req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:51.615126 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd52252f24114465af9bdf6e6fe68b76', 'request_id': 'req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:51.624622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd52252f24114465af9bdf6e6fe68b76', 'request_id': 'req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:51.637883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] The token cache is being invalidate because project cd52252f24114465af9bdf6e6fe68b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:45:51.638177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:51.638960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4c2fb3a3-c360-4da7-8cec-8b175fb4690a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:51.640833 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1015/2040] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:45:51 2022] DELETE /identity/v3/projects/cd52252f24114465af9bdf6e6fe68b76 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:51.645508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be624f2-11ea-44fc-af4d-dc49ac88e43f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:51.645920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be624f2-11ea-44fc-af4d-dc49ac88e43f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:51.646287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be624f2-11ea-44fc-af4d-dc49ac88e43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:51.666525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-4be624f2-11ea-44fc-af4d-dc49ac88e43f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:51.667740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4be624f2-11ea-44fc-af4d-dc49ac88e43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:51.686134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:51.731555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1026/2041] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:45:51.739788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae88118-8224-41bd-ba89-576bca5d7e33 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:51.740078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae88118-8224-41bd-ba89-576bca5d7e33 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:51.740353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae88118-8224-41bd-ba89-576bca5d7e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:51.760485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-6ae88118-8224-41bd-ba89-576bca5d7e33 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:51.761755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6ae88118-8224-41bd-ba89-576bca5d7e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:51.770003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1016/2042] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:51.774502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be624f2-11ea-44fc-af4d-dc49ac88e43f None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:51.784379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be624f2-11ea-44fc-af4d-dc49ac88e43f None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:51.789756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119a182-b358-4084-b0ec-82eeb1483fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:51.790362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a182-b358-4084-b0ec-82eeb1483fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:51.790645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a182-b358-4084-b0ec-82eeb1483fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:51.790918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a182-b358-4084-b0ec-82eeb1483fcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:51.791568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119a182-b358-4084-b0ec-82eeb1483fcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:45:51.791873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119a182-b358-4084-b0ec-82eeb1483fcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:51.793510 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119a182-b358-4084-b0ec-82eeb1483fcf admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:51.805964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1027/2043] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:45:51.810400 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae88118-8224-41bd-ba89-576bca5d7e33 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:51.819867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae88118-8224-41bd-ba89-576bca5d7e33 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:51.824802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ba3ad-a429-450f-b956-15addb6c5dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:51.825445 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ba3ad-a429-450f-b956-15addb6c5dde admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:51.825727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ba3ad-a429-450f-b956-15addb6c5dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:51.826003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ba3ad-a429-450f-b956-15addb6c5dde admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:51.826647 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ba3ad-a429-450f-b956-15addb6c5dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:51.828552 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ba3ad-a429-450f-b956-15addb6c5dde admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:51.860403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1017/2044] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:45:51 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:45:51.861823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119a182-b358-4084-b0ec-82eeb1483fcf admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:51.872527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119a182-b358-4084-b0ec-82eeb1483fcf admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:51.930316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670214a7-85c1-4b90-ad24-19df64212d10 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:51.930691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670214a7-85c1-4b90-ad24-19df64212d10 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:51.930939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670214a7-85c1-4b90-ad24-19df64212d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:51.942197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efb947-f0f9-417d-8403-1d7f16a986ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:51.942849 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efb947-f0f9-417d-8403-1d7f16a986ee admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:51.943135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efb947-f0f9-417d-8403-1d7f16a986ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:51.943406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efb947-f0f9-417d-8403-1d7f16a986ee admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:51.944058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efb947-f0f9-417d-8403-1d7f16a986ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:51.946115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efb947-f0f9-417d-8403-1d7f16a986ee admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:51.958642 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-670214a7-85c1-4b90-ad24-19df64212d10 None None] MFA Rules not processed for user `916699a212eb4d8aa945f299ac0d4c96`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:51.959964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-670214a7-85c1-4b90-ad24-19df64212d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:51.974118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1028/2045] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:45:51 2022] POST /identity/v3/users => generated 303 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:45:51.975488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efb947-f0f9-417d-8403-1d7f16a986ee admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:51.986601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efb947-f0f9-417d-8403-1d7f16a986ee admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:51.992037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae756c-7a1b-4b1b-a415-0eed7fda9804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:51.992690 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae756c-7a1b-4b1b-a415-0eed7fda9804 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:51.993007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae756c-7a1b-4b1b-a415-0eed7fda9804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:51.993275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae756c-7a1b-4b1b-a415-0eed7fda9804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:51.993917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae756c-7a1b-4b1b-a415-0eed7fda9804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:51.995652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae756c-7a1b-4b1b-a415-0eed7fda9804 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.007641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1029/2046] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:45:52.011167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:52.012083 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae756c-7a1b-4b1b-a415-0eed7fda9804 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.024691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae756c-7a1b-4b1b-a415-0eed7fda9804 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.029615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66fd6a-7a5e-479f-b117-5eb81d99a600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.030215 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fd6a-7a5e-479f-b117-5eb81d99a600 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.030494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fd6a-7a5e-479f-b117-5eb81d99a600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.030953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66fd6a-7a5e-479f-b117-5eb81d99a600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.031091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1018/2047] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:52.031612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66fd6a-7a5e-479f-b117-5eb81d99a600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.033273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66fd6a-7a5e-479f-b117-5eb81d99a600 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.045501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1030/2048] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:45:52.049951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670214a7-85c1-4b90-ad24-19df64212d10 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.062360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670214a7-85c1-4b90-ad24-19df64212d10 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.067651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bd4fd-7dce-419f-a533-9ac7f52ee846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.067767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bd4fd-7dce-419f-a533-9ac7f52ee846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.068003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bd4fd-7dce-419f-a533-9ac7f52ee846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.068275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bd4fd-7dce-419f-a533-9ac7f52ee846 admin admin] PATH_INFO: `/v3/projects/0b77deddd94c4fd9a7685b28a920b1a2/users/c9541042308744e493bda6cc5b096a3b/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.084129 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4bd4fd-7dce-419f-a533-9ac7f52ee846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b77deddd94c4fd9a7685b28a920b1a2, user_id=c9541042308744e493bda6cc5b096a3b, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.086869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4bd4fd-7dce-419f-a533-9ac7f52ee846 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.100802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1019/2049] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:45:52 2022] PUT /identity/v3/projects/0b77deddd94c4fd9a7685b28a920b1a2/users/c9541042308744e493bda6cc5b096a3b/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:52.111138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89ded0-4a2f-475c-ad7b-255883ee35d8 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.111426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89ded0-4a2f-475c-ad7b-255883ee35d8 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.111704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89ded0-4a2f-475c-ad7b-255883ee35d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.140970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-7b89ded0-4a2f-475c-ad7b-255883ee35d8 None None] MFA Rules not processed for user `c9541042308744e493bda6cc5b096a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:52.142972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7b89ded0-4a2f-475c-ad7b-255883ee35d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:52.218229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1031/2050] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:52 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:52.222761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bd4fd-7dce-419f-a533-9ac7f52ee846 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.285148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bd4fd-7dce-419f-a533-9ac7f52ee846 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.290765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e24ba-d3c7-4397-8b03-b1e5f8cb7755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.291418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e24ba-d3c7-4397-8b03-b1e5f8cb7755 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.291695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e24ba-d3c7-4397-8b03-b1e5f8cb7755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.291968 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e24ba-d3c7-4397-8b03-b1e5f8cb7755 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.292633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e24ba-d3c7-4397-8b03-b1e5f8cb7755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.294606 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e24ba-d3c7-4397-8b03-b1e5f8cb7755 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.322796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1020/2051] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:45:52 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:45:52.327024 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89ded0-4a2f-475c-ad7b-255883ee35d8 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.339339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89ded0-4a2f-475c-ad7b-255883ee35d8 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.343661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:52.358976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e24ba-d3c7-4397-8b03-b1e5f8cb7755 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.361368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c78e24ba-d3c7-4397-8b03-b1e5f8cb7755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:52.401666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e43640-2c81-446c-be69-6edddc166150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.402480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e43640-2c81-446c-be69-6edddc166150 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.402761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e43640-2c81-446c-be69-6edddc166150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.403030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e43640-2c81-446c-be69-6edddc166150 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.403649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e43640-2c81-446c-be69-6edddc166150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.405723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e43640-2c81-446c-be69-6edddc166150 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.421814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e24ba-d3c7-4397-8b03-b1e5f8cb7755 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.426007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d68512-9fe0-4ce3-8088-b43d0d0c95a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.426632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d68512-9fe0-4ce3-8088-b43d0d0c95a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.426934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d68512-9fe0-4ce3-8088-b43d0d0c95a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.427207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d68512-9fe0-4ce3-8088-b43d0d0c95a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.428999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-08d68512-9fe0-4ce3-8088-b43d0d0c95a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:52.432477 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1032/2052] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:45:52 2022] POST /identity/v3/users => generated 309 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:45:52.433756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e43640-2c81-446c-be69-6edddc166150 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.439631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d68512-9fe0-4ce3-8088-b43d0d0c95a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.441114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d68512-9fe0-4ce3-8088-b43d0d0c95a3 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.446452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e43640-2c81-446c-be69-6edddc166150 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.450029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1021/2053] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:45:52.451473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c18054-2952-4b9b-93ac-b0bb5a4d18ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.452158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c18054-2952-4b9b-93ac-b0bb5a4d18ca admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.452506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c18054-2952-4b9b-93ac-b0bb5a4d18ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.452929 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c18054-2952-4b9b-93ac-b0bb5a4d18ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.453535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c18054-2952-4b9b-93ac-b0bb5a4d18ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.455136 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c18054-2952-4b9b-93ac-b0bb5a4d18ca admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.466812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1033/2054] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:45:52.471014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d68512-9fe0-4ce3-8088-b43d0d0c95a3 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.487909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d68512-9fe0-4ce3-8088-b43d0d0c95a3 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.492602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e51da8-c717-4425-b6a3-0fe8602ec723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.493327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e51da8-c717-4425-b6a3-0fe8602ec723 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.493600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e51da8-c717-4425-b6a3-0fe8602ec723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.493866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e51da8-c717-4425-b6a3-0fe8602ec723 admin admin] PATH_INFO: `/v3/projects/6a40d517282b4a2cb5aeaf1957c89c58/users/ba4dede57e8944719bbebb6ef09b74d4/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.509555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e51da8-c717-4425-b6a3-0fe8602ec723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a40d517282b4a2cb5aeaf1957c89c58, user_id=ba4dede57e8944719bbebb6ef09b74d4, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.512205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e51da8-c717-4425-b6a3-0fe8602ec723 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.526964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1022/2055] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:45:52 2022] PUT /identity/v3/projects/6a40d517282b4a2cb5aeaf1957c89c58/users/ba4dede57e8944719bbebb6ef09b74d4/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:52.530986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c18054-2952-4b9b-93ac-b0bb5a4d18ca admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.540921 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c18054-2952-4b9b-93ac-b0bb5a4d18ca admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.599795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22f0f0-a199-4b47-b9d5-ae52c95c3f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.600441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22f0f0-a199-4b47-b9d5-ae52c95c3f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.600720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22f0f0-a199-4b47-b9d5-ae52c95c3f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.601022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22f0f0-a199-4b47-b9d5-ae52c95c3f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.601672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22f0f0-a199-4b47-b9d5-ae52c95c3f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.603268 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22f0f0-a199-4b47-b9d5-ae52c95c3f3a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.613511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1034/2056] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:45:52.617807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e51da8-c717-4425-b6a3-0fe8602ec723 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.627949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e51da8-c717-4425-b6a3-0fe8602ec723 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.633236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4913-1204-4324-aef0-60baf1ab52b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.633889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4913-1204-4324-aef0-60baf1ab52b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.634174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4913-1204-4324-aef0-60baf1ab52b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.634428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4913-1204-4324-aef0-60baf1ab52b7 admin admin] PATH_INFO: `/v3/projects/6a40d517282b4a2cb5aeaf1957c89c58/users/ba4dede57e8944719bbebb6ef09b74d4/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.636330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb4913-1204-4324-aef0-60baf1ab52b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a40d517282b4a2cb5aeaf1957c89c58, user_id=ba4dede57e8944719bbebb6ef09b74d4, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.639020 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb4913-1204-4324-aef0-60baf1ab52b7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.650408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1023/2057] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:45:52 2022] PUT /identity/v3/projects/6a40d517282b4a2cb5aeaf1957c89c58/users/ba4dede57e8944719bbebb6ef09b74d4/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:52.662314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3e521-71eb-49a0-84e9-7112253cdda1 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.662601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3e521-71eb-49a0-84e9-7112253cdda1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.662878 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3e521-71eb-49a0-84e9-7112253cdda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.691715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-83d3e521-71eb-49a0-84e9-7112253cdda1 None None] MFA Rules not processed for user `ba4dede57e8944719bbebb6ef09b74d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:52.693458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-83d3e521-71eb-49a0-84e9-7112253cdda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:52.764778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1035/2058] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:52 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:52.776682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4913-1204-4324-aef0-60baf1ab52b7 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.778287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-99eb4913-1204-4324-aef0-60baf1ab52b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:52.782019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3e521-71eb-49a0-84e9-7112253cdda1 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:52.783473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-83d3e521-71eb-49a0-84e9-7112253cdda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:52.794908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:52.802958 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:52.853906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4913-1204-4324-aef0-60baf1ab52b7 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.858479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a8867-fdf7-4f56-a4a2-8b3968e6f3f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.859130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a8867-fdf7-4f56-a4a2-8b3968e6f3f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.859404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a8867-fdf7-4f56-a4a2-8b3968e6f3f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.859674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a8867-fdf7-4f56-a4a2-8b3968e6f3f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.861745 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3e521-71eb-49a0-84e9-7112253cdda1 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:52.868124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c5232-c521-487c-85c4-bd2913fe61e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:52.868735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c5232-c521-487c-85c4-bd2913fe61e2 service placement] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:52.869046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c5232-c521-487c-85c4-bd2913fe61e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:52.869319 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c5232-c521-487c-85c4-bd2913fe61e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:52.869446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a8867-fdf7-4f56-a4a2-8b3968e6f3f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.870867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a8867-fdf7-4f56-a4a2-8b3968e6f3f5 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.871055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-100c5232-c521-487c-85c4-bd2913fe61e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:52.880692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1024/2059] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:52 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:45:52.883358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c5232-c521-487c-85c4-bd2913fe61e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:52.885058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c5232-c521-487c-85c4-bd2913fe61e2 service placement] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:52.895929 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1036/2060] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:45:53.028600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a8867-fdf7-4f56-a4a2-8b3968e6f3f5 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:53.094442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a8867-fdf7-4f56-a4a2-8b3968e6f3f5 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:53.100062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:53.100691 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:53.101292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:53.101570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] PATH_INFO: `/v3/users/c9541042308744e493bda6cc5b096a3b` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:53.103172 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9541042308744e493bda6cc5b096a3b)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:53.105394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:53.155915 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9541042308744e493bda6cc5b096a3b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.165648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9541042308744e493bda6cc5b096a3b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.166515 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9541042308744e493bda6cc5b096a3b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.180831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9541042308744e493bda6cc5b096a3b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.189812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9541042308744e493bda6cc5b096a3b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.196059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1025/2061] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:45:53 2022] DELETE /identity/v3/users/c9541042308744e493bda6cc5b096a3b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:53.201212 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2ffc-1ada-45db-8933-3bcc9cd0c01c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:53.201607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2ffc-1ada-45db-8933-3bcc9cd0c01c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:53.201890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2ffc-1ada-45db-8933-3bcc9cd0c01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:53.220966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-adcf2ffc-1ada-45db-8933-3bcc9cd0c01c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:53.222164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-adcf2ffc-1ada-45db-8933-3bcc9cd0c01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:53.285560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1037/2062] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:53.296444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:53.297931 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:53.346765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:53.362758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a604c93-b9f8-4180-b295-99bddc7d1fb6 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:53.367238 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3e16d-aa4e-4c3d-80d7-529fb3712809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:53.367917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3e16d-aa4e-4c3d-80d7-529fb3712809 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:53.368260 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3e16d-aa4e-4c3d-80d7-529fb3712809 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:53.368534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3e16d-aa4e-4c3d-80d7-529fb3712809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:53.378498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3e16d-aa4e-4c3d-80d7-529fb3712809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:53.380009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3e16d-aa4e-4c3d-80d7-529fb3712809 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:53.390135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1026/2063] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:45:53.670381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf2ffc-1ada-45db-8933-3bcc9cd0c01c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:53.671809 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-adcf2ffc-1ada-45db-8933-3bcc9cd0c01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:53.688857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf2ffc-1ada-45db-8933-3bcc9cd0c01c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:53.693567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:53.694202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:53.694482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:53.694838 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] PATH_INFO: `/v3/projects/0b77deddd94c4fd9a7685b28a920b1a2` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:53.695938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b77deddd94c4fd9a7685b28a920b1a2)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:53.697804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:53.744780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b77deddd94c4fd9a7685b28a920b1a2', 'request_id': 'req-793376d3-53c5-4211-b8ff-7f07725780be'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.750240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b77deddd94c4fd9a7685b28a920b1a2', 'request_id': 'req-793376d3-53c5-4211-b8ff-7f07725780be'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.757516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b77deddd94c4fd9a7685b28a920b1a2', 'request_id': 'req-793376d3-53c5-4211-b8ff-7f07725780be'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.768980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] The token cache is being invalidate because project 0b77deddd94c4fd9a7685b28a920b1a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:45:53.769268 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.769892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.771910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1038/2064] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:45:53 2022] DELETE /identity/v3/projects/0b77deddd94c4fd9a7685b28a920b1a2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:53.775923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3e16d-aa4e-4c3d-80d7-529fb3712809 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:53.787364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3e16d-aa4e-4c3d-80d7-529fb3712809 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:53.850750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:53.851387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:53.851657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:53.851924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] PATH_INFO: `/v3/users/ba4dede57e8944719bbebb6ef09b74d4` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:53.853064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba4dede57e8944719bbebb6ef09b74d4)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:53.854971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:53.911660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba4dede57e8944719bbebb6ef09b74d4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.919272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba4dede57e8944719bbebb6ef09b74d4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.920000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba4dede57e8944719bbebb6ef09b74d4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.936190 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba4dede57e8944719bbebb6ef09b74d4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.950885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a4abe79d-bdb5-4009-9b9d-e2e1344e210b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba4dede57e8944719bbebb6ef09b74d4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:53.959318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1027/2065] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:45:53 2022] DELETE /identity/v3/users/ba4dede57e8944719bbebb6ef09b74d4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:54.249702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:54.251142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:54.265005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793376d3-53c5-4211-b8ff-7f07725780be admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:54.291742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:54.335063 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:54.336341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:54.336341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:54.336341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] PATH_INFO: `/v3/projects/6a40d517282b4a2cb5aeaf1957c89c58` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:54.337626 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a40d517282b4a2cb5aeaf1957c89c58)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:54.339824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:54.393698 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a40d517282b4a2cb5aeaf1957c89c58', 'request_id': 'req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:54.400200 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a40d517282b4a2cb5aeaf1957c89c58', 'request_id': 'req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:54.408685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a40d517282b4a2cb5aeaf1957c89c58', 'request_id': 'req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:54.421776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] The token cache is being invalidate because project 6a40d517282b4a2cb5aeaf1957c89c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:45:54.421980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:54.422647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a7d4bdc2-56e2-4c57-a8c6-aa7140e8b1d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:45:54.424175 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1039/2066] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:45:54 2022] DELETE /identity/v3/projects/6a40d517282b4a2cb5aeaf1957c89c58 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:54.429842 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655f74c-90cc-4a9e-bdbd-6e8458ecb0f5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:54.430132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655f74c-90cc-4a9e-bdbd-6e8458ecb0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:54.430455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655f74c-90cc-4a9e-bdbd-6e8458ecb0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:54.451777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e655f74c-90cc-4a9e-bdbd-6e8458ecb0f5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:54.453057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e655f74c-90cc-4a9e-bdbd-6e8458ecb0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:54.518339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1028/2067] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:54.526026 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382ab3d-d459-4193-a7cd-aa7d37040425 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:54.526316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382ab3d-d459-4193-a7cd-aa7d37040425 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:54.526596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382ab3d-d459-4193-a7cd-aa7d37040425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:54.547099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d382ab3d-d459-4193-a7cd-aa7d37040425 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:54.548305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d382ab3d-d459-4193-a7cd-aa7d37040425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:54.556696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1040/2068] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:54.561036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655f74c-90cc-4a9e-bdbd-6e8458ecb0f5 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:54.565958 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:54.573484 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655f74c-90cc-4a9e-bdbd-6e8458ecb0f5 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:54.579493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394f6a-f758-46ab-ac7f-2da3d929339f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:54.580231 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394f6a-f758-46ab-ac7f-2da3d929339f admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:54.580572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394f6a-f758-46ab-ac7f-2da3d929339f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:54.581191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394f6a-f758-46ab-ac7f-2da3d929339f admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:54.581910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394f6a-f758-46ab-ac7f-2da3d929339f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:45:54.582274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394f6a-f758-46ab-ac7f-2da3d929339f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:54.583971 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394f6a-f758-46ab-ac7f-2da3d929339f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:54.596256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1029/2069] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:45:54.600473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382ab3d-d459-4193-a7cd-aa7d37040425 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:54.609913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382ab3d-d459-4193-a7cd-aa7d37040425 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:54.615092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10f899-6919-4bec-ae38-27418d93a8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:54.615689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10f899-6919-4bec-ae38-27418d93a8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:54.615979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10f899-6919-4bec-ae38-27418d93a8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:54.616231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10f899-6919-4bec-ae38-27418d93a8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:54.616921 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10f899-6919-4bec-ae38-27418d93a8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:54.618823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10f899-6919-4bec-ae38-27418d93a8e3 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:54.649253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1041/2070] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:45:54 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:45:54.653562 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394f6a-f758-46ab-ac7f-2da3d929339f admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:54.664742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394f6a-f758-46ab-ac7f-2da3d929339f admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:54.725932 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8039d9-26ee-4f72-bbe9-2271f6a5902f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:54.726556 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8039d9-26ee-4f72-bbe9-2271f6a5902f admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:54.726830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8039d9-26ee-4f72-bbe9-2271f6a5902f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:54.727406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8039d9-26ee-4f72-bbe9-2271f6a5902f admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:54.727736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8039d9-26ee-4f72-bbe9-2271f6a5902f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:54.729781 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8039d9-26ee-4f72-bbe9-2271f6a5902f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:54.754769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1030/2071] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:45:54 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:45:54.755853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10f899-6919-4bec-ae38-27418d93a8e3 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:54.766206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10f899-6919-4bec-ae38-27418d93a8e3 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:54.770804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1de55-ec10-475f-868c-ab57ac625e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:54.771397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1de55-ec10-475f-868c-ab57ac625e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:54.771671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1de55-ec10-475f-868c-ab57ac625e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:54.771934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1de55-ec10-475f-868c-ab57ac625e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:54.772563 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1de55-ec10-475f-868c-ab57ac625e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:54.774190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1de55-ec10-475f-868c-ab57ac625e54 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:54.788874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1042/2072] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:45:54.792817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8039d9-26ee-4f72-bbe9-2271f6a5902f admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:54.804376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8039d9-26ee-4f72-bbe9-2271f6a5902f admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:54.809514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3322fd6-ca7c-48d0-944f-8cfc8f99d1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:54.810182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3322fd6-ca7c-48d0-944f-8cfc8f99d1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:54.810458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3322fd6-ca7c-48d0-944f-8cfc8f99d1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:54.810724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3322fd6-ca7c-48d0-944f-8cfc8f99d1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:54.811363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3322fd6-ca7c-48d0-944f-8cfc8f99d1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:54.812981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3322fd6-ca7c-48d0-944f-8cfc8f99d1a5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:54.825248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1031/2073] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:45:54.829518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1de55-ec10-475f-868c-ab57ac625e54 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:54.840435 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1de55-ec10-475f-868c-ab57ac625e54 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:54.845732 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f041-ef6d-4ffe-a727-414ff746ae1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:54.846396 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f041-ef6d-4ffe-a727-414ff746ae1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:54.846672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f041-ef6d-4ffe-a727-414ff746ae1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:54.846939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f041-ef6d-4ffe-a727-414ff746ae1b admin admin] PATH_INFO: `/v3/projects/c63dc14265164f2a8ef7ba05374372c8/users/31413e201f594eb18468cd0784b7d6c2/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:54.862772 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4f041-ef6d-4ffe-a727-414ff746ae1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63dc14265164f2a8ef7ba05374372c8, user_id=31413e201f594eb18468cd0784b7d6c2, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:54.866254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4f041-ef6d-4ffe-a727-414ff746ae1b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:54.879585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1043/2074] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:45:54 2022] PUT /identity/v3/projects/c63dc14265164f2a8ef7ba05374372c8/users/31413e201f594eb18468cd0784b7d6c2/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:45:54.891407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0373a82-40d0-41a4-9d19-eb9af6aece7c None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:54.891875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0373a82-40d0-41a4-9d19-eb9af6aece7c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:54.892236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0373a82-40d0-41a4-9d19-eb9af6aece7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:54.920656 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-f0373a82-40d0-41a4-9d19-eb9af6aece7c None None] MFA Rules not processed for user `31413e201f594eb18468cd0784b7d6c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:54.922030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f0373a82-40d0-41a4-9d19-eb9af6aece7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:54.997189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1032/2075] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:55.003526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f041-ef6d-4ffe-a727-414ff746ae1b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:55.083009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:55.101204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f041-ef6d-4ffe-a727-414ff746ae1b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:55.111548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab512d-06c6-43c9-a736-58a9fcfa2f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:55.112275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab512d-06c6-43c9-a736-58a9fcfa2f05 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:55.112618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab512d-06c6-43c9-a736-58a9fcfa2f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:55.112990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab512d-06c6-43c9-a736-58a9fcfa2f05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:55.113808 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab512d-06c6-43c9-a736-58a9fcfa2f05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:55.116042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab512d-06c6-43c9-a736-58a9fcfa2f05 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:55.143561 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0373a82-40d0-41a4-9d19-eb9af6aece7c None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:55.146114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f0373a82-40d0-41a4-9d19-eb9af6aece7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:55.155657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1044/2076] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:45:55 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:45:55.159979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab512d-06c6-43c9-a736-58a9fcfa2f05 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:55.249417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0373a82-40d0-41a4-9d19-eb9af6aece7c None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:55.261482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:55.273871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab512d-06c6-43c9-a736-58a9fcfa2f05 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:55.280821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fed73c-c033-4ebd-9e39-19a3957bb174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:55.281771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fed73c-c033-4ebd-9e39-19a3957bb174 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:55.282199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fed73c-c033-4ebd-9e39-19a3957bb174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:55.282647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fed73c-c033-4ebd-9e39-19a3957bb174 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:55.283491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fed73c-c033-4ebd-9e39-19a3957bb174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:55.285680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fed73c-c033-4ebd-9e39-19a3957bb174 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:55.311127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1045/2077] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:45:55 2022] POST /identity/v3/users => generated 295 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:45:55.315035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fed73c-c033-4ebd-9e39-19a3957bb174 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:55.325973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb939042-62d6-4d4b-8bff-a0b82bebbd67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:55.326205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fed73c-c033-4ebd-9e39-19a3957bb174 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:55.326736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb939042-62d6-4d4b-8bff-a0b82bebbd67 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:55.327086 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb939042-62d6-4d4b-8bff-a0b82bebbd67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:55.327418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb939042-62d6-4d4b-8bff-a0b82bebbd67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:55.329308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cb939042-62d6-4d4b-8bff-a0b82bebbd67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:55.331547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa89a3-6ab6-4163-a7ed-0787e493bb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:55.332238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa89a3-6ab6-4163-a7ed-0787e493bb12 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:55.333191 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa89a3-6ab6-4163-a7ed-0787e493bb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:55.333591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa89a3-6ab6-4163-a7ed-0787e493bb12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:55.334525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa89a3-6ab6-4163-a7ed-0787e493bb12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:55.336197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa89a3-6ab6-4163-a7ed-0787e493bb12 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:55.341528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb939042-62d6-4d4b-8bff-a0b82bebbd67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:55.343103 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb939042-62d6-4d4b-8bff-a0b82bebbd67 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:55.351447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1046/2078] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:45:55.354996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1033/2079] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:45:55.355716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa89a3-6ab6-4163-a7ed-0787e493bb12 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:55.366782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa89a3-6ab6-4163-a7ed-0787e493bb12 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:55.371524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d14319-3d6e-4195-af03-b9b37620c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:55.372185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d14319-3d6e-4195-af03-b9b37620c3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:55.372463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d14319-3d6e-4195-af03-b9b37620c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:55.372733 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d14319-3d6e-4195-af03-b9b37620c3f8 admin admin] PATH_INFO: `/v3/projects/bb836f4b622c441190220c46cc74da8a/users/69b6ba04350249cc808413c2d5145df2/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:55.388095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d14319-3d6e-4195-af03-b9b37620c3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb836f4b622c441190220c46cc74da8a, user_id=69b6ba04350249cc808413c2d5145df2, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:55.390781 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d14319-3d6e-4195-af03-b9b37620c3f8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:55.409995 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1047/2080] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:45:55 2022] PUT /identity/v3/projects/bb836f4b622c441190220c46cc74da8a/users/69b6ba04350249cc808413c2d5145df2/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:45:55.412310 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb939042-62d6-4d4b-8bff-a0b82bebbd67 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:55.424059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb939042-62d6-4d4b-8bff-a0b82bebbd67 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:55.483114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e195c-bfe1-4e36-9c34-e55a9580bcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:55.483726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e195c-bfe1-4e36-9c34-e55a9580bcd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:55.484002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e195c-bfe1-4e36-9c34-e55a9580bcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:55.484287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e195c-bfe1-4e36-9c34-e55a9580bcd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:55.484956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e195c-bfe1-4e36-9c34-e55a9580bcd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:55.486553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e195c-bfe1-4e36-9c34-e55a9580bcd4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:55.500344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1034/2081] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:45:55.501893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d14319-3d6e-4195-af03-b9b37620c3f8 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:55.512783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d14319-3d6e-4195-af03-b9b37620c3f8 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:55.517441 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75869a3-e018-4101-b636-d10a43e56ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:55.518090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75869a3-e018-4101-b636-d10a43e56ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:55.518374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75869a3-e018-4101-b636-d10a43e56ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:55.518637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75869a3-e018-4101-b636-d10a43e56ce2 admin admin] PATH_INFO: `/v3/projects/bb836f4b622c441190220c46cc74da8a/users/69b6ba04350249cc808413c2d5145df2/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:55.521192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75869a3-e018-4101-b636-d10a43e56ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb836f4b622c441190220c46cc74da8a, user_id=69b6ba04350249cc808413c2d5145df2, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:55.523823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75869a3-e018-4101-b636-d10a43e56ce2 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:55.534760 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1048/2082] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:45:55 2022] PUT /identity/v3/projects/bb836f4b622c441190220c46cc74da8a/users/69b6ba04350249cc808413c2d5145df2/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:45:55.544858 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fdb7a-6790-4320-ae88-73620d179f5f None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:55.545147 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fdb7a-6790-4320-ae88-73620d179f5f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:55.545415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fdb7a-6790-4320-ae88-73620d179f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:55.573624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-0f5fdb7a-6790-4320-ae88-73620d179f5f None None] MFA Rules not processed for user `69b6ba04350249cc808413c2d5145df2`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:45:55.574849 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0f5fdb7a-6790-4320-ae88-73620d179f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:55.651830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1035/2083] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:45:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:45:55.662440 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75869a3-e018-4101-b636-d10a43e56ce2 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:55.664339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-c75869a3-e018-4101-b636-d10a43e56ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:55.725554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75869a3-e018-4101-b636-d10a43e56ce2 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:55.729738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7c095-8d5f-47ef-99be-daf0beade72e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:55.730450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7c095-8d5f-47ef-99be-daf0beade72e service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:55.730737 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7c095-8d5f-47ef-99be-daf0beade72e service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:55.731010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7c095-8d5f-47ef-99be-daf0beade72e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:55.759417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:55.794307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7c095-8d5f-47ef-99be-daf0beade72e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:55.795766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7c095-8d5f-47ef-99be-daf0beade72e service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:55.805642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1049/2084] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:45:57.738509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fdb7a-6790-4320-ae88-73620d179f5f None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:45:57.739910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0f5fdb7a-6790-4320-ae88-73620d179f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:57.783821 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:45:57.817558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fdb7a-6790-4320-ae88-73620d179f5f None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:45:57.823125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7350204-5fd1-4bf7-b257-a12bcbf4d082 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:45:57.823809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7350204-5fd1-4bf7-b257-a12bcbf4d082 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:45:57.824006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7350204-5fd1-4bf7-b257-a12bcbf4d082 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:45:57.824278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7350204-5fd1-4bf7-b257-a12bcbf4d082 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:45:57.826041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f7350204-5fd1-4bf7-b257-a12bcbf4d082 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:45:57.886604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7350204-5fd1-4bf7-b257-a12bcbf4d082 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:45:57.888236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7350204-5fd1-4bf7-b257-a12bcbf4d082 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:45:57.899530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1036/2085] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:45:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:01.500795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7c095-8d5f-47ef-99be-daf0beade72e service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:01.509568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7c095-8d5f-47ef-99be-daf0beade72e service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:01.514038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd419f0-8ffd-486b-987b-30d609c11f78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:01.514701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd419f0-8ffd-486b-987b-30d609c11f78 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:01.514982 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd419f0-8ffd-486b-987b-30d609c11f78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:01.515252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd419f0-8ffd-486b-987b-30d609c11f78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:01.525167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd419f0-8ffd-486b-987b-30d609c11f78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:01.526591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd419f0-8ffd-486b-987b-30d609c11f78 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:01.538816 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1050/2086] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:07.894224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=39 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:08.076297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7350204-5fd1-4bf7-b257-a12bcbf4d082 service nova] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:08.088097 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7350204-5fd1-4bf7-b257-a12bcbf4d082 service nova] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:08.093715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea6077-9694-4cbe-8ab0-3127a6113367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:08.094318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea6077-9694-4cbe-8ab0-3127a6113367 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:08.094590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea6077-9694-4cbe-8ab0-3127a6113367 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:08.094857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea6077-9694-4cbe-8ab0-3127a6113367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:08.097008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0aea6077-9694-4cbe-8ab0-3127a6113367 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:08.113946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea6077-9694-4cbe-8ab0-3127a6113367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:08.115545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea6077-9694-4cbe-8ab0-3127a6113367 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:08.127883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1037/2087] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:11.532107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=21 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:18.122288 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=6 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:18.152391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9957c8-914d-45a6-bb47-00d2fd95daa5 None None] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:18.153137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9957c8-914d-45a6-bb47-00d2fd95daa5 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:18.153137 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9957c8-914d-45a6-bb47-00d2fd95daa5 None None] PATH_INFO: `/` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:18.154517 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1051/2088] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 18 11:46:18.173969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b118105-cf88-4997-995d-c7f4bf1a830e None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:18.174434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b118105-cf88-4997-995d-c7f4bf1a830e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:18.176899 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b118105-cf88-4997-995d-c7f4bf1a830e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:18.202824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-0b118105-cf88-4997-995d-c7f4bf1a830e None None] MFA Rules not processed for user `d301f3d9aeb644a38c31dbdbef2a48c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:18.204045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0b118105-cf88-4997-995d-c7f4bf1a830e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:18.268394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1038/2089] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:46:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:18.289775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9957c8-914d-45a6-bb47-00d2fd95daa5 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:18.299714 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5f9957c8-914d-45a6-bb47-00d2fd95daa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:18.312937 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9957c8-914d-45a6-bb47-00d2fd95daa5 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:18.316622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef97b54-ef94-416e-89ae-6d69d56d399a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:18.317264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef97b54-ef94-416e-89ae-6d69d56d399a service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:18.317681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef97b54-ef94-416e-89ae-6d69d56d399a service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:18.318000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef97b54-ef94-416e-89ae-6d69d56d399a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:18.325581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef97b54-ef94-416e-89ae-6d69d56d399a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:18.327002 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef97b54-ef94-416e-89ae-6d69d56d399a service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:18.336505 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1052/2090] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:18.423373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871fd2a-ef1c-4379-a15f-9bfde384d23d None None] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:18.423805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871fd2a-ef1c-4379-a15f-9bfde384d23d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:18.424093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871fd2a-ef1c-4379-a15f-9bfde384d23d None None] PATH_INFO: `/` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:18.425542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1039/2091] 10.222.0.17 () {54 vars in 921 bytes} [Tue Oct 18 11:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 18 11:46:18.431353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92945034-c392-45c6-aee9-b5e8062dde98 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:18.431661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92945034-c392-45c6-aee9-b5e8062dde98 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:18.431932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92945034-c392-45c6-aee9-b5e8062dde98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:18.452323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-92945034-c392-45c6-aee9-b5e8062dde98 None None] MFA Rules not processed for user `87f8a4a8277745a9ad1976af9df31cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:18.453933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-92945034-c392-45c6-aee9-b5e8062dde98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:18.464315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1053/2092] 10.222.0.17 () {58 vars in 1018 bytes} [Tue Oct 18 11:46:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:18.478070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871fd2a-ef1c-4379-a15f-9bfde384d23d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:18.490315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871fd2a-ef1c-4379-a15f-9bfde384d23d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:18.494543 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c016e5-3df3-437a-8ef7-a11f2e51b2ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:18.495505 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c016e5-3df3-437a-8ef7-a11f2e51b2ea service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:18.495720 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c016e5-3df3-437a-8ef7-a11f2e51b2ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:18.496084 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c016e5-3df3-437a-8ef7-a11f2e51b2ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:18.511583 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c016e5-3df3-437a-8ef7-a11f2e51b2ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:18.513315 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c016e5-3df3-437a-8ef7-a11f2e51b2ea service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:18.526874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1040/2093] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:20.808778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92945034-c392-45c6-aee9-b5e8062dde98 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:20.820975 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92945034-c392-45c6-aee9-b5e8062dde98 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:20.827164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a90ca4-c82a-4067-ac9d-332bec23fdf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:20.827850 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a90ca4-c82a-4067-ac9d-332bec23fdf7 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:20.828201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a90ca4-c82a-4067-ac9d-332bec23fdf7 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:20.828540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a90ca4-c82a-4067-ac9d-332bec23fdf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:20.830615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-32a90ca4-c82a-4067-ac9d-332bec23fdf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:20.843029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a90ca4-c82a-4067-ac9d-332bec23fdf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:20.844632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a90ca4-c82a-4067-ac9d-332bec23fdf7 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:20.855988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1054/2094] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:26.702572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c016e5-3df3-437a-8ef7-a11f2e51b2ea service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:26.768361 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c016e5-3df3-437a-8ef7-a11f2e51b2ea service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:26.773542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:26.774164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:26.774444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:26.774712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] PATH_INFO: `/v3/users/31413e201f594eb18468cd0784b7d6c2` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:26.775844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31413e201f594eb18468cd0784b7d6c2)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:26.777803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:26.822885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31413e201f594eb18468cd0784b7d6c2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:26.830066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31413e201f594eb18468cd0784b7d6c2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:26.830724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31413e201f594eb18468cd0784b7d6c2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:26.842791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31413e201f594eb18468cd0784b7d6c2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:26.850447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31413e201f594eb18468cd0784b7d6c2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:26.856537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1041/2095] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:26 2022] DELETE /identity/v3/users/31413e201f594eb18468cd0784b7d6c2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:26.861059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542303d-e4db-4c7a-b3b4-7ebdf3edbf46 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:26.861546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542303d-e4db-4c7a-b3b4-7ebdf3edbf46 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:26.861643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542303d-e4db-4c7a-b3b4-7ebdf3edbf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:26.881515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-a542303d-e4db-4c7a-b3b4-7ebdf3edbf46 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:26.882732 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a542303d-e4db-4c7a-b3b4-7ebdf3edbf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:26.943771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1055/2096] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:26.955862 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:26.957300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:26.978087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:27.026244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ea865-fa9d-42fb-9c90-2a5165214696 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:27.034564 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d801f-f5af-46e1-8adc-ac3cc2251832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:27.035250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d801f-f5af-46e1-8adc-ac3cc2251832 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:27.035527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d801f-f5af-46e1-8adc-ac3cc2251832 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:27.035801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d801f-f5af-46e1-8adc-ac3cc2251832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:27.045999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d801f-f5af-46e1-8adc-ac3cc2251832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:27.047457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d801f-f5af-46e1-8adc-ac3cc2251832 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:27.057467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1042/2097] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:27.343878 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542303d-e4db-4c7a-b3b4-7ebdf3edbf46 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:27.345521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a542303d-e4db-4c7a-b3b4-7ebdf3edbf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:27.359890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542303d-e4db-4c7a-b3b4-7ebdf3edbf46 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:27.364694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:27.365580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:27.366048 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:27.366533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] PATH_INFO: `/v3/projects/c63dc14265164f2a8ef7ba05374372c8` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:27.368306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c63dc14265164f2a8ef7ba05374372c8)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:27.370264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:27.421086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c63dc14265164f2a8ef7ba05374372c8', 'request_id': 'req-5dedc774-5618-4c7a-bef6-075a0e201064'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:27.427261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c63dc14265164f2a8ef7ba05374372c8', 'request_id': 'req-5dedc774-5618-4c7a-bef6-075a0e201064'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:27.436713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c63dc14265164f2a8ef7ba05374372c8', 'request_id': 'req-5dedc774-5618-4c7a-bef6-075a0e201064'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:27.450397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] The token cache is being invalidate because project c63dc14265164f2a8ef7ba05374372c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:27.450712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:27.451372 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:27.453076 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1056/2098] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:27 2022] DELETE /identity/v3/projects/c63dc14265164f2a8ef7ba05374372c8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:27.457170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d801f-f5af-46e1-8adc-ac3cc2251832 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:27.467974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d801f-f5af-46e1-8adc-ac3cc2251832 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:27.529871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:27.530540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:27.530825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:27.531100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] PATH_INFO: `/v3/users/69b6ba04350249cc808413c2d5145df2` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:27.532503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=69b6ba04350249cc808413c2d5145df2)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:27.534706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:27.589809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69b6ba04350249cc808413c2d5145df2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:27.597004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69b6ba04350249cc808413c2d5145df2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:27.597805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69b6ba04350249cc808413c2d5145df2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:27.611899 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69b6ba04350249cc808413c2d5145df2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:27.620354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-38a0e6d2-75d6-4484-90ad-db23196114ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69b6ba04350249cc808413c2d5145df2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:27.627123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1043/2099] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:27 2022] DELETE /identity/v3/users/69b6ba04350249cc808413c2d5145df2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:28.001869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.003349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:28.021345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedc774-5618-4c7a-bef6-075a0e201064 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.072656 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:28.092282 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.092958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.093238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.093512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] PATH_INFO: `/v3/projects/bb836f4b622c441190220c46cc74da8a` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.094653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb836f4b622c441190220c46cc74da8a)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:28.096600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:28.144278 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb836f4b622c441190220c46cc74da8a', 'request_id': 'req-57685898-499f-4ae3-afe3-9a0fb196f8b7'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:28.150694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb836f4b622c441190220c46cc74da8a', 'request_id': 'req-57685898-499f-4ae3-afe3-9a0fb196f8b7'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:28.158316 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb836f4b622c441190220c46cc74da8a', 'request_id': 'req-57685898-499f-4ae3-afe3-9a0fb196f8b7'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:28.171629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] The token cache is being invalidate because project bb836f4b622c441190220c46cc74da8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:28.171915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:28.172592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-57685898-499f-4ae3-afe3-9a0fb196f8b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:28.174100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1057/2100] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:28 2022] DELETE /identity/v3/projects/bb836f4b622c441190220c46cc74da8a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:28.179486 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cdd23-1302-4028-97e8-c1f956a36421 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.180245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cdd23-1302-4028-97e8-c1f956a36421 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.180577 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cdd23-1302-4028-97e8-c1f956a36421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.200693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-307cdd23-1302-4028-97e8-c1f956a36421 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:28.201903 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-307cdd23-1302-4028-97e8-c1f956a36421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:28.248333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:28.278922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1044/2101] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:28.289114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef944a-0a78-453f-ad03-03263fb537e9 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.289404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef944a-0a78-453f-ad03-03263fb537e9 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.289673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef944a-0a78-453f-ad03-03263fb537e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.310324 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-7aef944a-0a78-453f-ad03-03263fb537e9 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:28.311472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7aef944a-0a78-453f-ad03-03263fb537e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:28.319364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1058/2102] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:46:28.323878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307cdd23-1302-4028-97e8-c1f956a36421 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.334570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307cdd23-1302-4028-97e8-c1f956a36421 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.339181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f669c-8580-4d06-8ad7-60c457849504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.339785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f669c-8580-4d06-8ad7-60c457849504 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.340065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f669c-8580-4d06-8ad7-60c457849504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.340339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f669c-8580-4d06-8ad7-60c457849504 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.341067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f669c-8580-4d06-8ad7-60c457849504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:46:28.341370 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f669c-8580-4d06-8ad7-60c457849504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:28.342963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f669c-8580-4d06-8ad7-60c457849504 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:28.355261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1045/2103] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:46:28.359723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef944a-0a78-453f-ad03-03263fb537e9 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.368858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef944a-0a78-453f-ad03-03263fb537e9 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.374229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab735-c8b7-460c-8caf-c0da83a06482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.374828 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab735-c8b7-460c-8caf-c0da83a06482 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.375134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab735-c8b7-460c-8caf-c0da83a06482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.375405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab735-c8b7-460c-8caf-c0da83a06482 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.376031 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab735-c8b7-460c-8caf-c0da83a06482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:28.377970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab735-c8b7-460c-8caf-c0da83a06482 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:28.406082 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1059/2104] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:28 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:28.410302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f669c-8580-4d06-8ad7-60c457849504 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.420224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f669c-8580-4d06-8ad7-60c457849504 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.476871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b293a82-466a-4874-a906-3cf1970b1045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.477475 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293a82-466a-4874-a906-3cf1970b1045 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.477750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293a82-466a-4874-a906-3cf1970b1045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.478018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293a82-466a-4874-a906-3cf1970b1045 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.478649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b293a82-466a-4874-a906-3cf1970b1045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:28.480541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b293a82-466a-4874-a906-3cf1970b1045 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:28.504301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1046/2105] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:28 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:28.509088 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab735-c8b7-460c-8caf-c0da83a06482 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.519372 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab735-c8b7-460c-8caf-c0da83a06482 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.524505 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4af603-c2bf-428e-b037-452327913fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.525130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4af603-c2bf-428e-b037-452327913fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.525406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4af603-c2bf-428e-b037-452327913fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.525675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4af603-c2bf-428e-b037-452327913fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.526330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4af603-c2bf-428e-b037-452327913fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:28.527906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4af603-c2bf-428e-b037-452327913fe3 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:28.537890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1060/2106] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:28.541798 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b293a82-466a-4874-a906-3cf1970b1045 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.551711 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b293a82-466a-4874-a906-3cf1970b1045 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.556295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239006-fc4e-48f2-82ad-7d0699c77a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.556903 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239006-fc4e-48f2-82ad-7d0699c77a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.557181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239006-fc4e-48f2-82ad-7d0699c77a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.557448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239006-fc4e-48f2-82ad-7d0699c77a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.558078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239006-fc4e-48f2-82ad-7d0699c77a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:28.559653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239006-fc4e-48f2-82ad-7d0699c77a53 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:28.570612 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1047/2107] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:28.574593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4af603-c2bf-428e-b037-452327913fe3 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.585422 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4af603-c2bf-428e-b037-452327913fe3 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.590016 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63aeec0-228a-424a-bf5b-15d9a6e0cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.590654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aeec0-228a-424a-bf5b-15d9a6e0cd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.590933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aeec0-228a-424a-bf5b-15d9a6e0cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.591203 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aeec0-228a-424a-bf5b-15d9a6e0cd91 admin admin] PATH_INFO: `/v3/projects/864004cf07e24ee9a49196654ae96760/users/45b2f559166842a4bc813aa74837a0f9/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.606112 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63aeec0-228a-424a-bf5b-15d9a6e0cd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=864004cf07e24ee9a49196654ae96760, user_id=45b2f559166842a4bc813aa74837a0f9, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:28.608770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63aeec0-228a-424a-bf5b-15d9a6e0cd91 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:28.621147 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1061/2108] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:28 2022] PUT /identity/v3/projects/864004cf07e24ee9a49196654ae96760/users/45b2f559166842a4bc813aa74837a0f9/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:28.631826 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9fb92-fc57-41cb-9ad4-305867ba935a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.632112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9fb92-fc57-41cb-9ad4-305867ba935a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.632384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9fb92-fc57-41cb-9ad4-305867ba935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.661104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d7c9fb92-fc57-41cb-9ad4-305867ba935a None None] MFA Rules not processed for user `45b2f559166842a4bc813aa74837a0f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:28.662132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d7c9fb92-fc57-41cb-9ad4-305867ba935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:28.729430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1048/2109] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:28.734043 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63aeec0-228a-424a-bf5b-15d9a6e0cd91 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.795949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63aeec0-228a-424a-bf5b-15d9a6e0cd91 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.800731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561365e-0d44-480d-b6c0-49d6716e750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.801418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561365e-0d44-480d-b6c0-49d6716e750a admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.802204 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561365e-0d44-480d-b6c0-49d6716e750a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.802473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561365e-0d44-480d-b6c0-49d6716e750a admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.803313 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561365e-0d44-480d-b6c0-49d6716e750a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:28.805040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561365e-0d44-480d-b6c0-49d6716e750a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:28.833483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1062/2110] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:28 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:28.837576 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9fb92-fc57-41cb-9ad4-305867ba935a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.848242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9fb92-fc57-41cb-9ad4-305867ba935a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.880542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:28.910029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279940c9-c5a1-487d-b1ee-9a4ea59f921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.910636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279940c9-c5a1-487d-b1ee-9a4ea59f921f admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.910908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279940c9-c5a1-487d-b1ee-9a4ea59f921f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.911205 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279940c9-c5a1-487d-b1ee-9a4ea59f921f admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.911829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279940c9-c5a1-487d-b1ee-9a4ea59f921f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:28.913781 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279940c9-c5a1-487d-b1ee-9a4ea59f921f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:28.940637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1049/2111] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:28 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:46:28.941926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561365e-0d44-480d-b6c0-49d6716e750a admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.948346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:28.954424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561365e-0d44-480d-b6c0-49d6716e750a admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.961600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be4d37-5b11-4da9-ac42-24c5280ab057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.962240 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4d37-5b11-4da9-ac42-24c5280ab057 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.962521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4d37-5b11-4da9-ac42-24c5280ab057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.962794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4d37-5b11-4da9-ac42-24c5280ab057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:28.963437 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be4d37-5b11-4da9-ac42-24c5280ab057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:28.965050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be4d37-5b11-4da9-ac42-24c5280ab057 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:28.975889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1063/2112] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:46:28.980298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279940c9-c5a1-487d-b1ee-9a4ea59f921f admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:28.990934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279940c9-c5a1-487d-b1ee-9a4ea59f921f admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:28.995831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b54fe3-d880-4b66-97ab-f4ead4ff1a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:28.996496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b54fe3-d880-4b66-97ab-f4ead4ff1a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:28.996773 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b54fe3-d880-4b66-97ab-f4ead4ff1a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:28.997086 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b54fe3-d880-4b66-97ab-f4ead4ff1a39 admin admin] PATH_INFO: `/v3/projects/7c3d418a8a834d07af9e2fde34e16b41/users/91991f5b89054575bf3992c66fb9f202/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:29.013207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b54fe3-d880-4b66-97ab-f4ead4ff1a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c3d418a8a834d07af9e2fde34e16b41, user_id=91991f5b89054575bf3992c66fb9f202, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:29.015929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b54fe3-d880-4b66-97ab-f4ead4ff1a39 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:29.039415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1050/2113] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:28 2022] PUT /identity/v3/projects/7c3d418a8a834d07af9e2fde34e16b41/users/91991f5b89054575bf3992c66fb9f202/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:46:29.043448 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be4d37-5b11-4da9-ac42-24c5280ab057 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:29.054152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be4d37-5b11-4da9-ac42-24c5280ab057 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:29.112135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aaa1a6-57f9-4acc-bc5d-a2cee7d44ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:29.112657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aaa1a6-57f9-4acc-bc5d-a2cee7d44ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:29.112975 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aaa1a6-57f9-4acc-bc5d-a2cee7d44ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:29.113251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aaa1a6-57f9-4acc-bc5d-a2cee7d44ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:29.113898 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aaa1a6-57f9-4acc-bc5d-a2cee7d44ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:29.116089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aaa1a6-57f9-4acc-bc5d-a2cee7d44ce2 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:29.129090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1064/2114] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:29.130346 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b54fe3-d880-4b66-97ab-f4ead4ff1a39 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:29.140600 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b54fe3-d880-4b66-97ab-f4ead4ff1a39 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:29.145213 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17152b-cb96-48b6-a461-b27a2dd0ca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:29.145871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17152b-cb96-48b6-a461-b27a2dd0ca3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:29.146156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17152b-cb96-48b6-a461-b27a2dd0ca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:29.146422 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17152b-cb96-48b6-a461-b27a2dd0ca3d admin admin] PATH_INFO: `/v3/projects/7c3d418a8a834d07af9e2fde34e16b41/users/91991f5b89054575bf3992c66fb9f202/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:29.148355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17152b-cb96-48b6-a461-b27a2dd0ca3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c3d418a8a834d07af9e2fde34e16b41, user_id=91991f5b89054575bf3992c66fb9f202, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:29.151046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17152b-cb96-48b6-a461-b27a2dd0ca3d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:29.163589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1051/2115] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:29 2022] PUT /identity/v3/projects/7c3d418a8a834d07af9e2fde34e16b41/users/91991f5b89054575bf3992c66fb9f202/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:29.174542 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ebd6c-327c-4f46-b64c-8b8244c10192 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:29.174862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ebd6c-327c-4f46-b64c-8b8244c10192 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:29.175095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ebd6c-327c-4f46-b64c-8b8244c10192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:29.208002 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-948ebd6c-327c-4f46-b64c-8b8244c10192 None None] MFA Rules not processed for user `91991f5b89054575bf3992c66fb9f202`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:29.209280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-948ebd6c-327c-4f46-b64c-8b8244c10192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:29.283470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1065/2116] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:29.294988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17152b-cb96-48b6-a461-b27a2dd0ca3d admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:29.296748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fa17152b-cb96-48b6-a461-b27a2dd0ca3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:29.353994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17152b-cb96-48b6-a461-b27a2dd0ca3d admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:29.357682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9808136-13e2-4cbb-b637-a6294e05d1e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:29.358268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9808136-13e2-4cbb-b637-a6294e05d1e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:29.358542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9808136-13e2-4cbb-b637-a6294e05d1e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:29.358811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9808136-13e2-4cbb-b637-a6294e05d1e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:29.368286 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9808136-13e2-4cbb-b637-a6294e05d1e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:29.369746 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9808136-13e2-4cbb-b637-a6294e05d1e2 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:29.379013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1052/2117] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:29.407628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ebd6c-327c-4f46-b64c-8b8244c10192 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:29.470853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ebd6c-327c-4f46-b64c-8b8244c10192 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:29.476349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:29.477025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:29.477306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:29.477581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] PATH_INFO: `/v3/users/45b2f559166842a4bc813aa74837a0f9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:29.478776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] RBAC: Authorizing `identity:delete_user(user_id=45b2f559166842a4bc813aa74837a0f9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:29.480739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:29.495540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:29.528222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45b2f559166842a4bc813aa74837a0f9'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:29.540873 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45b2f559166842a4bc813aa74837a0f9'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:29.541610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45b2f559166842a4bc813aa74837a0f9'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:29.548498 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45b2f559166842a4bc813aa74837a0f9'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:29.549357 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45b2f559166842a4bc813aa74837a0f9'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:29.563538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1066/2118] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:29 2022] DELETE /identity/v3/users/45b2f559166842a4bc813aa74837a0f9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:29.568193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9f79a-d309-4555-97a5-bcfc9370c26a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:29.568475 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9f79a-d309-4555-97a5-bcfc9370c26a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:29.568744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9f79a-d309-4555-97a5-bcfc9370c26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:29.588430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-cfd9f79a-d309-4555-97a5-bcfc9370c26a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:29.589654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cfd9f79a-d309-4555-97a5-bcfc9370c26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:29.646965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1053/2119] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:29.656776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:29.658172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:29.721874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0f27c-1c2e-4773-abdf-bd18a395183b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:29.726177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dead6-e7f9-4426-8ccd-2f09896252bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:29.726794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dead6-e7f9-4426-8ccd-2f09896252bb service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:29.727070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dead6-e7f9-4426-8ccd-2f09896252bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:29.727394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dead6-e7f9-4426-8ccd-2f09896252bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:29.737579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dead6-e7f9-4426-8ccd-2f09896252bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:29.739121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dead6-e7f9-4426-8ccd-2f09896252bb service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:29.760926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1067/2120] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:30.244532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9f79a-d309-4555-97a5-bcfc9370c26a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:30.246703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cfd9f79a-d309-4555-97a5-bcfc9370c26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:30.265624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9f79a-d309-4555-97a5-bcfc9370c26a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:30.279757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:30.280538 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:30.280823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:30.281117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] PATH_INFO: `/v3/projects/864004cf07e24ee9a49196654ae96760` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:30.284693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=864004cf07e24ee9a49196654ae96760)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:30.287645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:30.334015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:30.364208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '864004cf07e24ee9a49196654ae96760', 'request_id': 'req-6ee72e89-93dd-4a08-8132-50cdf6f89693'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:30.372021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '864004cf07e24ee9a49196654ae96760', 'request_id': 'req-6ee72e89-93dd-4a08-8132-50cdf6f89693'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:30.382118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '864004cf07e24ee9a49196654ae96760', 'request_id': 'req-6ee72e89-93dd-4a08-8132-50cdf6f89693'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:30.397244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] The token cache is being invalidate because project 864004cf07e24ee9a49196654ae96760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:30.397681 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:30.398451 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ee72e89-93dd-4a08-8132-50cdf6f89693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:30.400912 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1054/2121] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:30 2022] DELETE /identity/v3/projects/864004cf07e24ee9a49196654ae96760 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:30.404822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dead6-e7f9-4426-8ccd-2f09896252bb service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:30.437000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dead6-e7f9-4426-8ccd-2f09896252bb service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:30.506008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e1d67-b6b4-4abf-bff1-cf587854e065 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:30.506535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e1d67-b6b4-4abf-bff1-cf587854e065 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:30.507030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e1d67-b6b4-4abf-bff1-cf587854e065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:30.516232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:30.516897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:30.517183 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:30.517458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] PATH_INFO: `/v3/users/91991f5b89054575bf3992c66fb9f202` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:30.518595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91991f5b89054575bf3992c66fb9f202)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:30.521558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:30.535499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-8f4e1d67-b6b4-4abf-bff1-cf587854e065 None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:30.537469 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8f4e1d67-b6b4-4abf-bff1-cf587854e065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:30.569895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:30.583936 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91991f5b89054575bf3992c66fb9f202'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:30.599755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91991f5b89054575bf3992c66fb9f202'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:30.600513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91991f5b89054575bf3992c66fb9f202'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:30.605745 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1055/2122] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:46:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:30.608404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91991f5b89054575bf3992c66fb9f202'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:30.609711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91991f5b89054575bf3992c66fb9f202'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:30.623841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e1d67-b6b4-4abf-bff1-cf587854e065 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:30.626818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8f4e1d67-b6b4-4abf-bff1-cf587854e065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:30.636326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1068/2123] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:30 2022] DELETE /identity/v3/users/91991f5b89054575bf3992c66fb9f202 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:30.723954 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e1d67-b6b4-4abf-bff1-cf587854e065 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:30.793436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:30.810040 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49f131-bdfc-4e66-b719-9c1fa721002f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:30.810678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f131-bdfc-4e66-b719-9c1fa721002f service swift] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:30.811341 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f131-bdfc-4e66-b719-9c1fa721002f service swift] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:30.811982 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49f131-bdfc-4e66-b719-9c1fa721002f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:30.814139 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6f49f131-bdfc-4e66-b719-9c1fa721002f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:30.882499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49f131-bdfc-4e66-b719-9c1fa721002f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:30.884098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49f131-bdfc-4e66-b719-9c1fa721002f service swift] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:30.892766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1056/2124] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:46:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:46:31.166380 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:31.173400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:31.371942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567647e-be59-471f-a881-f176d32a1655 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:31.383981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:31.384579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:31.384939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:31.385163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] PATH_INFO: `/v3/projects/7c3d418a8a834d07af9e2fde34e16b41` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:31.386328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c3d418a8a834d07af9e2fde34e16b41)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:31.388261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:31.417810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49f131-bdfc-4e66-b719-9c1fa721002f service swift] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:31.420108 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6f49f131-bdfc-4e66-b719-9c1fa721002f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:31.505662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c3d418a8a834d07af9e2fde34e16b41', 'request_id': 'req-e25b519b-dd68-4272-b84b-36ca07603f58'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:31.521506 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c3d418a8a834d07af9e2fde34e16b41', 'request_id': 'req-e25b519b-dd68-4272-b84b-36ca07603f58'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:31.539490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c3d418a8a834d07af9e2fde34e16b41', 'request_id': 'req-e25b519b-dd68-4272-b84b-36ca07603f58'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:31.562231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] The token cache is being invalidate because project 7c3d418a8a834d07af9e2fde34e16b41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:31.565760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49f131-bdfc-4e66-b719-9c1fa721002f service swift] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:31.566696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:31.570986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e25b519b-dd68-4272-b84b-36ca07603f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:31.577326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1069/2125] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:31 2022] DELETE /identity/v3/projects/7c3d418a8a834d07af9e2fde34e16b41 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:31.581884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad5f33-2f05-4698-9754-f30e541edae7 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:31.582173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad5f33-2f05-4698-9754-f30e541edae7 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:31.582455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad5f33-2f05-4698-9754-f30e541edae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:31.616705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-1bad5f33-2f05-4698-9754-f30e541edae7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:31.618387 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1bad5f33-2f05-4698-9754-f30e541edae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:31.673841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b719a3-3395-490f-a44b-27bbc3273ba0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:31.674430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b719a3-3395-490f-a44b-27bbc3273ba0 service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:31.674748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b719a3-3395-490f-a44b-27bbc3273ba0 service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:31.675023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b719a3-3395-490f-a44b-27bbc3273ba0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:31.678041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f8b719a3-3395-490f-a44b-27bbc3273ba0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:31.695345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b719a3-3395-490f-a44b-27bbc3273ba0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:31.697227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b719a3-3395-490f-a44b-27bbc3273ba0 service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:31.715267 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1057/2126] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:31.720067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:31.760612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1070/2127] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:31.762148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bd0a6-6eab-434f-92fd-34e8a690ac92 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:31.762816 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bd0a6-6eab-434f-92fd-34e8a690ac92 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:31.763162 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bd0a6-6eab-434f-92fd-34e8a690ac92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:31.820397 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-163bd0a6-6eab-434f-92fd-34e8a690ac92 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:31.824900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-163bd0a6-6eab-434f-92fd-34e8a690ac92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:31.839168 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1058/2128] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:31.843687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad5f33-2f05-4698-9754-f30e541edae7 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:31.855954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad5f33-2f05-4698-9754-f30e541edae7 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:31.862699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2c952-a18a-4d0d-b450-82a8d84755c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:31.864093 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2c952-a18a-4d0d-b450-82a8d84755c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:31.865044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2c952-a18a-4d0d-b450-82a8d84755c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:31.865583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2c952-a18a-4d0d-b450-82a8d84755c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:31.867301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2c952-a18a-4d0d-b450-82a8d84755c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:46:31.867661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2c952-a18a-4d0d-b450-82a8d84755c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:31.870305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2c952-a18a-4d0d-b450-82a8d84755c8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:31.920054 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1071/2129] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:46:31.920891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bd0a6-6eab-434f-92fd-34e8a690ac92 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:31.944961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bd0a6-6eab-434f-92fd-34e8a690ac92 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:31.959523 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bcd52-f132-4c3c-9849-23b5810f2270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:31.961309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bcd52-f132-4c3c-9849-23b5810f2270 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:31.961761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bcd52-f132-4c3c-9849-23b5810f2270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:31.962220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bcd52-f132-4c3c-9849-23b5810f2270 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:31.963048 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bcd52-f132-4c3c-9849-23b5810f2270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:31.966444 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bcd52-f132-4c3c-9849-23b5810f2270 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:31.993634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:32.033204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1059/2130] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:31 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:46:32.045170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2c952-a18a-4d0d-b450-82a8d84755c8 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:32.062154 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2c952-a18a-4d0d-b450-82a8d84755c8 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:32.160255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdbe79-012a-4a5b-bcd7-b8fbdf965123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:32.161042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdbe79-012a-4a5b-bcd7-b8fbdf965123 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:32.161351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdbe79-012a-4a5b-bcd7-b8fbdf965123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:32.161665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdbe79-012a-4a5b-bcd7-b8fbdf965123 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:32.162383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdbe79-012a-4a5b-bcd7-b8fbdf965123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:32.164954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdbe79-012a-4a5b-bcd7-b8fbdf965123 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:32.209519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1072/2131] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:32 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:32.209994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bcd52-f132-4c3c-9849-23b5810f2270 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:32.230831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bcd52-f132-4c3c-9849-23b5810f2270 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:32.245760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db456156-84dd-4951-ab44-e8af1b6703d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:32.247335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db456156-84dd-4951-ab44-e8af1b6703d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:32.248024 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db456156-84dd-4951-ab44-e8af1b6703d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:32.248313 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db456156-84dd-4951-ab44-e8af1b6703d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:32.251185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db456156-84dd-4951-ab44-e8af1b6703d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:32.252835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db456156-84dd-4951-ab44-e8af1b6703d4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:32.275517 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1060/2132] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:32.280443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdbe79-012a-4a5b-bcd7-b8fbdf965123 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:32.300636 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdbe79-012a-4a5b-bcd7-b8fbdf965123 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:32.307590 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcb4ef-709a-4a70-859c-31f22627ca53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:32.308621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcb4ef-709a-4a70-859c-31f22627ca53 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:32.309149 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcb4ef-709a-4a70-859c-31f22627ca53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:32.309656 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcb4ef-709a-4a70-859c-31f22627ca53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:32.310521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fcb4ef-709a-4a70-859c-31f22627ca53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:32.312381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fcb4ef-709a-4a70-859c-31f22627ca53 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:32.328638 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1073/2133] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:32.331308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db456156-84dd-4951-ab44-e8af1b6703d4 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:32.348492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db456156-84dd-4951-ab44-e8af1b6703d4 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:32.357386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c4190-ce9c-4c9a-85fe-d6bfbdbb24c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:32.358299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c4190-ce9c-4c9a-85fe-d6bfbdbb24c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:32.358680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c4190-ce9c-4c9a-85fe-d6bfbdbb24c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:32.360291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c4190-ce9c-4c9a-85fe-d6bfbdbb24c1 admin admin] PATH_INFO: `/v3/projects/667ea9fd050e4247953c96d7f5c83790/users/348122e79f2545c6a2fa57c6972e6375/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:32.398140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c4190-ce9c-4c9a-85fe-d6bfbdbb24c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=667ea9fd050e4247953c96d7f5c83790, user_id=348122e79f2545c6a2fa57c6972e6375, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:32.401399 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c4190-ce9c-4c9a-85fe-d6bfbdbb24c1 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:32.464903 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1061/2134] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:32 2022] PUT /identity/v3/projects/667ea9fd050e4247953c96d7f5c83790/users/348122e79f2545c6a2fa57c6972e6375/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:32.471305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998749e-447f-4b4c-a5d3-ba1ffa0c66a5 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:32.471544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998749e-447f-4b4c-a5d3-ba1ffa0c66a5 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:32.471755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998749e-447f-4b4c-a5d3-ba1ffa0c66a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:32.532345 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-7998749e-447f-4b4c-a5d3-ba1ffa0c66a5 None None] MFA Rules not processed for user `348122e79f2545c6a2fa57c6972e6375`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:32.534056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7998749e-447f-4b4c-a5d3-ba1ffa0c66a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:32.622972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1074/2135] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:32.627680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c4190-ce9c-4c9a-85fe-d6bfbdbb24c1 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:32.704788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c4190-ce9c-4c9a-85fe-d6bfbdbb24c1 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:32.710801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feecdde-3fe9-4945-8b5d-1264d74ee56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:32.711429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feecdde-3fe9-4945-8b5d-1264d74ee56b admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:32.711852 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feecdde-3fe9-4945-8b5d-1264d74ee56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:32.712124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feecdde-3fe9-4945-8b5d-1264d74ee56b admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:32.712766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feecdde-3fe9-4945-8b5d-1264d74ee56b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:32.714779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feecdde-3fe9-4945-8b5d-1264d74ee56b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:32.752250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1062/2136] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:32 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:32.754134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998749e-447f-4b4c-a5d3-ba1ffa0c66a5 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:32.768588 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998749e-447f-4b4c-a5d3-ba1ffa0c66a5 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:32.816133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:32.844092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44aaf-7fd4-4512-9ddb-05a015f29a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:32.844731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44aaf-7fd4-4512-9ddb-05a015f29a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:32.845047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44aaf-7fd4-4512-9ddb-05a015f29a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:32.845348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44aaf-7fd4-4512-9ddb-05a015f29a7f admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:32.846031 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44aaf-7fd4-4512-9ddb-05a015f29a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:32.847975 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44aaf-7fd4-4512-9ddb-05a015f29a7f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:32.892413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1075/2137] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:32 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:32.895965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feecdde-3fe9-4945-8b5d-1264d74ee56b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:32.912268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feecdde-3fe9-4945-8b5d-1264d74ee56b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:32.919452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad44272-1676-40a8-b2d2-8eced228c87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:32.920354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad44272-1676-40a8-b2d2-8eced228c87b admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:32.920702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad44272-1676-40a8-b2d2-8eced228c87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:32.921116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad44272-1676-40a8-b2d2-8eced228c87b admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:32.921961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad44272-1676-40a8-b2d2-8eced228c87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:32.923683 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad44272-1676-40a8-b2d2-8eced228c87b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:32.951294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1063/2138] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:46:32.954925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44aaf-7fd4-4512-9ddb-05a015f29a7f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:32.971236 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44aaf-7fd4-4512-9ddb-05a015f29a7f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:32.983545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156b048-7c31-46cd-832f-311ddd78fd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:32.984293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156b048-7c31-46cd-832f-311ddd78fd00 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:32.984646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156b048-7c31-46cd-832f-311ddd78fd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:32.985672 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156b048-7c31-46cd-832f-311ddd78fd00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:32.986413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156b048-7c31-46cd-832f-311ddd78fd00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:32.989359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156b048-7c31-46cd-832f-311ddd78fd00 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:33.010631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad44272-1676-40a8-b2d2-8eced228c87b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:33.011793 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1076/2139] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:46:33.030586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad44272-1676-40a8-b2d2-8eced228c87b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:33.036233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1e436-b783-454a-b0fd-14983c2c808a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:33.037014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1e436-b783-454a-b0fd-14983c2c808a admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:33.037364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1e436-b783-454a-b0fd-14983c2c808a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:33.037707 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1e436-b783-454a-b0fd-14983c2c808a admin admin] PATH_INFO: `/v3/projects/1046024096a14f80b333657474cabc43/users/4347db990281464393128ca569353fef/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:33.092716 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1e436-b783-454a-b0fd-14983c2c808a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1046024096a14f80b333657474cabc43, user_id=4347db990281464393128ca569353fef, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:33.096802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1e436-b783-454a-b0fd-14983c2c808a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:33.130160 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1064/2140] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:33 2022] PUT /identity/v3/projects/1046024096a14f80b333657474cabc43/users/4347db990281464393128ca569353fef/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:33.141302 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02022ec-dd39-4412-ae1c-93fc38464316 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:33.141577 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02022ec-dd39-4412-ae1c-93fc38464316 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:33.141849 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02022ec-dd39-4412-ae1c-93fc38464316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:33.218016 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-f02022ec-dd39-4412-ae1c-93fc38464316 None None] MFA Rules not processed for user `4347db990281464393128ca569353fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:33.219555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f02022ec-dd39-4412-ae1c-93fc38464316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:33.357980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1077/2141] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:33.359385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1e436-b783-454a-b0fd-14983c2c808a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:33.447589 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1e436-b783-454a-b0fd-14983c2c808a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:33.452986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ae95a-f9ed-4fe2-a0b5-a0eda4ab3f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:33.453841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ae95a-f9ed-4fe2-a0b5-a0eda4ab3f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:33.454118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ae95a-f9ed-4fe2-a0b5-a0eda4ab3f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:33.454651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ae95a-f9ed-4fe2-a0b5-a0eda4ab3f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:33.455292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ae95a-f9ed-4fe2-a0b5-a0eda4ab3f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:33.457479 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ae95a-f9ed-4fe2-a0b5-a0eda4ab3f0b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:33.487939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:33.515921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1065/2142] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:33 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:33.521381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02022ec-dd39-4412-ae1c-93fc38464316 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:33.536977 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02022ec-dd39-4412-ae1c-93fc38464316 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:33.633716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97119dfc-1cee-4aba-9063-79b5348d50c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:33.635075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119dfc-1cee-4aba-9063-79b5348d50c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:33.635498 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119dfc-1cee-4aba-9063-79b5348d50c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:33.635877 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119dfc-1cee-4aba-9063-79b5348d50c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:33.636685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97119dfc-1cee-4aba-9063-79b5348d50c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:33.641438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97119dfc-1cee-4aba-9063-79b5348d50c6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:33.672139 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1078/2143] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:33 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:33.676342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ae95a-f9ed-4fe2-a0b5-a0eda4ab3f0b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:33.697801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ae95a-f9ed-4fe2-a0b5-a0eda4ab3f0b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:33.706964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e6cff-9862-4e92-8986-97de79f8ae91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:33.710628 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e6cff-9862-4e92-8986-97de79f8ae91 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:33.710809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e6cff-9862-4e92-8986-97de79f8ae91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:33.712166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e6cff-9862-4e92-8986-97de79f8ae91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:33.712748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e6cff-9862-4e92-8986-97de79f8ae91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:33.714498 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e6cff-9862-4e92-8986-97de79f8ae91 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:33.735483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97119dfc-1cee-4aba-9063-79b5348d50c6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:33.736008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1066/2144] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:46:33.748614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97119dfc-1cee-4aba-9063-79b5348d50c6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:33.755360 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f380-9e24-4843-82a1-01e72c84bc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:33.756386 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f380-9e24-4843-82a1-01e72c84bc95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:33.756667 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f380-9e24-4843-82a1-01e72c84bc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:33.756987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f380-9e24-4843-82a1-01e72c84bc95 admin admin] PATH_INFO: `/v3/projects/53a94452078e47f9aaed4973f95bbd14/users/04acecbb053b4d68bbafd233515d18db/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:33.780135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8f380-9e24-4843-82a1-01e72c84bc95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a94452078e47f9aaed4973f95bbd14, user_id=04acecbb053b4d68bbafd233515d18db, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:33.784041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8f380-9e24-4843-82a1-01e72c84bc95 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:33.823424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e6cff-9862-4e92-8986-97de79f8ae91 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:33.831414 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1079/2145] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:33 2022] PUT /identity/v3/projects/53a94452078e47f9aaed4973f95bbd14/users/04acecbb053b4d68bbafd233515d18db/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:46:33.839352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e6cff-9862-4e92-8986-97de79f8ae91 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:33.985042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3810b-4238-492f-8a1f-cb85c45cb1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:33.985951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3810b-4238-492f-8a1f-cb85c45cb1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:33.986390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3810b-4238-492f-8a1f-cb85c45cb1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:33.986815 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3810b-4238-492f-8a1f-cb85c45cb1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:33.987633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3810b-4238-492f-8a1f-cb85c45cb1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:33.989837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3810b-4238-492f-8a1f-cb85c45cb1b1 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:34.024548 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1067/2146] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:34.037655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f380-9e24-4843-82a1-01e72c84bc95 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:34.081363 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f380-9e24-4843-82a1-01e72c84bc95 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:34.094419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ca6cf-0f58-4206-bba7-8cbf5b5c13cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:34.097964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca6cf-0f58-4206-bba7-8cbf5b5c13cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:34.098451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca6cf-0f58-4206-bba7-8cbf5b5c13cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:34.099430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca6cf-0f58-4206-bba7-8cbf5b5c13cb admin admin] PATH_INFO: `/v3/projects/53a94452078e47f9aaed4973f95bbd14/users/04acecbb053b4d68bbafd233515d18db/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:34.104398 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ca6cf-0f58-4206-bba7-8cbf5b5c13cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a94452078e47f9aaed4973f95bbd14, user_id=04acecbb053b4d68bbafd233515d18db, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:34.110388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ca6cf-0f58-4206-bba7-8cbf5b5c13cb admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:34.123217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:34.145834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdfea4-3e95-4949-81a7-e89bf5fab4f5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:34.146125 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdfea4-3e95-4949-81a7-e89bf5fab4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:34.146403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdfea4-3e95-4949-81a7-e89bf5fab4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:34.146995 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1080/2147] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:34 2022] PUT /identity/v3/projects/53a94452078e47f9aaed4973f95bbd14/users/04acecbb053b4d68bbafd233515d18db/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:34.215614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-78fdfea4-3e95-4949-81a7-e89bf5fab4f5 None None] MFA Rules not processed for user `04acecbb053b4d68bbafd233515d18db`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:34.216869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-78fdfea4-3e95-4949-81a7-e89bf5fab4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:34.433433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1068/2148] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:34.445119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ca6cf-0f58-4206-bba7-8cbf5b5c13cb admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:34.447763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-610ca6cf-0f58-4206-bba7-8cbf5b5c13cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:34.643184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ca6cf-0f58-4206-bba7-8cbf5b5c13cb admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:34.663756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a80b4c-85a7-4529-852d-c27573cbdc00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:34.664490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a80b4c-85a7-4529-852d-c27573cbdc00 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:34.664893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a80b4c-85a7-4529-852d-c27573cbdc00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:34.665272 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a80b4c-85a7-4529-852d-c27573cbdc00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:34.686598 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a80b4c-85a7-4529-852d-c27573cbdc00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:34.688218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a80b4c-85a7-4529-852d-c27573cbdc00 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:34.710463 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1081/2149] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:34.777115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdfea4-3e95-4949-81a7-e89bf5fab4f5 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:34.778644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-78fdfea4-3e95-4949-81a7-e89bf5fab4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:34.799752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdfea4-3e95-4949-81a7-e89bf5fab4f5 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:34.809420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a10e2d-880a-44ad-a47e-e2965d5a00c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:34.810296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a10e2d-880a-44ad-a47e-e2965d5a00c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:34.820898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a10e2d-880a-44ad-a47e-e2965d5a00c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:34.820898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a10e2d-880a-44ad-a47e-e2965d5a00c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:34.911790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a10e2d-880a-44ad-a47e-e2965d5a00c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:34.914165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a10e2d-880a-44ad-a47e-e2965d5a00c7 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:34.938544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1069/2150] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:38.141456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a80b4c-85a7-4529-852d-c27573cbdc00 service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:38.221827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a80b4c-85a7-4529-852d-c27573cbdc00 service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:38.227924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:38.228720 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:38.229144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:38.229580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] PATH_INFO: `/v3/users/348122e79f2545c6a2fa57c6972e6375` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:38.241731 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=348122e79f2545c6a2fa57c6972e6375)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:38.244126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:38.315037 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348122e79f2545c6a2fa57c6972e6375'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:38.337996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348122e79f2545c6a2fa57c6972e6375'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:38.338967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348122e79f2545c6a2fa57c6972e6375'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:38.352027 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348122e79f2545c6a2fa57c6972e6375'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:38.353014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348122e79f2545c6a2fa57c6972e6375'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:38.384623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1082/2151] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:38 2022] DELETE /identity/v3/users/348122e79f2545c6a2fa57c6972e6375 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:38.389439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca893d8b-0246-4029-9b05-d2930d075dbd None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:38.390269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca893d8b-0246-4029-9b05-d2930d075dbd None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:38.390549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca893d8b-0246-4029-9b05-d2930d075dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:38.400249 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:38.413376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ca893d8b-0246-4029-9b05-d2930d075dbd None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:38.414801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ca893d8b-0246-4029-9b05-d2930d075dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:38.492408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1070/2152] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:38.502946 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:38.504528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:38.537520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:38.576768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fa833-168f-408f-9c13-733af947ca98 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:38.584907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606b617-fc26-476d-b253-c3f1b1cd0b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:38.584907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606b617-fc26-476d-b253-c3f1b1cd0b12 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:38.584907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606b617-fc26-476d-b253-c3f1b1cd0b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:38.584907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606b617-fc26-476d-b253-c3f1b1cd0b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:38.597596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606b617-fc26-476d-b253-c3f1b1cd0b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:38.599884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606b617-fc26-476d-b253-c3f1b1cd0b12 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:38.611563 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1083/2153] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:38.937778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca893d8b-0246-4029-9b05-d2930d075dbd None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:38.946655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606b617-fc26-476d-b253-c3f1b1cd0b12 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:38.948218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0606b617-fc26-476d-b253-c3f1b1cd0b12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:38.950897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca893d8b-0246-4029-9b05-d2930d075dbd None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:38.955750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307aede-a087-46d2-ae8c-51551a3d425a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:38.956740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307aede-a087-46d2-ae8c-51551a3d425a service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:38.957049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307aede-a087-46d2-ae8c-51551a3d425a service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:38.957368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307aede-a087-46d2-ae8c-51551a3d425a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:38.959745 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e307aede-a087-46d2-ae8c-51551a3d425a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:38.971366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606b617-fc26-476d-b253-c3f1b1cd0b12 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:38.986136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307aede-a087-46d2-ae8c-51551a3d425a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:38.987671 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307aede-a087-46d2-ae8c-51551a3d425a service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:38.988244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:38.988981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:38.989260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:38.994351 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] PATH_INFO: `/v3/projects/667ea9fd050e4247953c96d7f5c83790` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:38.995585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=667ea9fd050e4247953c96d7f5c83790)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:38.997722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:39.002005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1071/2154] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:46:39.071843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '667ea9fd050e4247953c96d7f5c83790', 'request_id': 'req-00da5f5d-34df-485b-b161-fd5f5e14fc5c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:39.078039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '667ea9fd050e4247953c96d7f5c83790', 'request_id': 'req-00da5f5d-34df-485b-b161-fd5f5e14fc5c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:39.091929 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '667ea9fd050e4247953c96d7f5c83790', 'request_id': 'req-00da5f5d-34df-485b-b161-fd5f5e14fc5c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:39.106812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] The token cache is being invalidate because project 667ea9fd050e4247953c96d7f5c83790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:39.107604 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:39.109326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:39.111010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1084/2155] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:38 2022] DELETE /identity/v3/projects/667ea9fd050e4247953c96d7f5c83790 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:39.115629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307aede-a087-46d2-ae8c-51551a3d425a service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:39.126667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307aede-a087-46d2-ae8c-51551a3d425a service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:39.207266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:39.208461 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:39.208731 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:39.209887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] PATH_INFO: `/v3/users/4347db990281464393128ca569353fef` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:39.211164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4347db990281464393128ca569353fef)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:39.214366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:39.310939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4347db990281464393128ca569353fef'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:39.322307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4347db990281464393128ca569353fef'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:39.322736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4347db990281464393128ca569353fef'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:39.338534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4347db990281464393128ca569353fef'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:39.347297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4347db990281464393128ca569353fef'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:39.355867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1072/2156] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:39 2022] DELETE /identity/v3/users/4347db990281464393128ca569353fef => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:46:39.851823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:39.853605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:39.893605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da5f5d-34df-485b-b161-fd5f5e14fc5c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:39.990036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:39.990698 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:39.990981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:39.991257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] PATH_INFO: `/v3/projects/1046024096a14f80b333657474cabc43` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:39.992415 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1046024096a14f80b333657474cabc43)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:39.994517 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:40.052471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1046024096a14f80b333657474cabc43', 'request_id': 'req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:40.053840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:40.059529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1046024096a14f80b333657474cabc43', 'request_id': 'req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:40.085427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1046024096a14f80b333657474cabc43', 'request_id': 'req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:40.114361 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] The token cache is being invalidate because project 1046024096a14f80b333657474cabc43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:40.114984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:40.115837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:40.121404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1085/2157] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:39 2022] DELETE /identity/v3/projects/1046024096a14f80b333657474cabc43 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:40.121830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:40.123398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:40.142409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65317ad-f389-41eb-8873-a0dbaafe3b22 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:40.153804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:40.251661 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:40.252325 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:40.252608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:40.252903 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] PATH_INFO: `/v3/users/04acecbb053b4d68bbafd233515d18db` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:40.255443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] RBAC: Authorizing `identity:delete_user(user_id=04acecbb053b4d68bbafd233515d18db)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:40.258345 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:40.332715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04acecbb053b4d68bbafd233515d18db'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:40.346051 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04acecbb053b4d68bbafd233515d18db'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:40.346853 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04acecbb053b4d68bbafd233515d18db'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:40.372382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04acecbb053b4d68bbafd233515d18db'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:40.385416 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6ba57ccd-6ef5-4e28-afc8-d66c152a234a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04acecbb053b4d68bbafd233515d18db'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:40.393521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1073/2158] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:40 2022] DELETE /identity/v3/users/04acecbb053b4d68bbafd233515d18db => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:46:40.778483 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:40.779857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:40.800375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aed2c6-d3e3-4ffa-bbe1-d33ee72f1b63 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:40.889440 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:40.891550 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:40.892216 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:40.892654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] PATH_INFO: `/v3/projects/53a94452078e47f9aaed4973f95bbd14` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:40.894367 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a94452078e47f9aaed4973f95bbd14)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:40.897164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:40.997220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a94452078e47f9aaed4973f95bbd14', 'request_id': 'req-658227a4-ec99-4b30-8908-ec2b81dd7ea0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:41.009159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a94452078e47f9aaed4973f95bbd14', 'request_id': 'req-658227a4-ec99-4b30-8908-ec2b81dd7ea0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:41.022688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a94452078e47f9aaed4973f95bbd14', 'request_id': 'req-658227a4-ec99-4b30-8908-ec2b81dd7ea0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:41.045250 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] The token cache is being invalidate because project 53a94452078e47f9aaed4973f95bbd14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:41.045692 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:41.046523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-658227a4-ec99-4b30-8908-ec2b81dd7ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:41.049478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1086/2159] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:40 2022] DELETE /identity/v3/projects/53a94452078e47f9aaed4973f95bbd14 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:41.069247 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d130a4-48d6-42e0-ae4c-568848a3e38f None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:41.071299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d130a4-48d6-42e0-ae4c-568848a3e38f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:41.071299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d130a4-48d6-42e0-ae4c-568848a3e38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:41.097391 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-80d130a4-48d6-42e0-ae4c-568848a3e38f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:41.099165 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-80d130a4-48d6-42e0-ae4c-568848a3e38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:41.209926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1074/2160] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:41.212371 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a041f60-4230-4c51-a697-3a2d057d12db None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:41.212658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a041f60-4230-4c51-a697-3a2d057d12db None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:41.212956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a041f60-4230-4c51-a697-3a2d057d12db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:41.267072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-4a041f60-4230-4c51-a697-3a2d057d12db None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:41.268776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4a041f60-4230-4c51-a697-3a2d057d12db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:41.288970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1087/2161] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:41.290275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d130a4-48d6-42e0-ae4c-568848a3e38f None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:41.303181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d130a4-48d6-42e0-ae4c-568848a3e38f None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:41.316620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d4a88-77af-494c-b79d-5f5eb82fc001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:41.317282 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4a88-77af-494c-b79d-5f5eb82fc001 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:41.317555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4a88-77af-494c-b79d-5f5eb82fc001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:41.317824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4a88-77af-494c-b79d-5f5eb82fc001 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:41.318533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d4a88-77af-494c-b79d-5f5eb82fc001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:46:41.318834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d4a88-77af-494c-b79d-5f5eb82fc001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:41.320811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d4a88-77af-494c-b79d-5f5eb82fc001 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:41.350859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1075/2162] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:46:41.355295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a041f60-4230-4c51-a697-3a2d057d12db None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:41.370667 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a041f60-4230-4c51-a697-3a2d057d12db None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:41.390582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9e17a-5418-472a-8bca-e6c9b7cc67ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:41.393136 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9e17a-5418-472a-8bca-e6c9b7cc67ae admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:41.395534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9e17a-5418-472a-8bca-e6c9b7cc67ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:41.398508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9e17a-5418-472a-8bca-e6c9b7cc67ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:41.399354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9e17a-5418-472a-8bca-e6c9b7cc67ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:41.403553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9e17a-5418-472a-8bca-e6c9b7cc67ae admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:41.470918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1088/2163] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:41 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:41.475491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d4a88-77af-494c-b79d-5f5eb82fc001 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:41.507199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d4a88-77af-494c-b79d-5f5eb82fc001 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:41.544132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:41.669077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5f5a-f7a3-4077-9985-4fca44c337b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:41.669944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5f5a-f7a3-4077-9985-4fca44c337b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:41.670424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5f5a-f7a3-4077-9985-4fca44c337b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:41.670839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5f5a-f7a3-4077-9985-4fca44c337b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:41.671877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d5f5a-f7a3-4077-9985-4fca44c337b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:41.674405 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d5f5a-f7a3-4077-9985-4fca44c337b2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:41.726676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1076/2164] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:41 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:41.731344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9e17a-5418-472a-8bca-e6c9b7cc67ae admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:41.750070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9e17a-5418-472a-8bca-e6c9b7cc67ae admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:41.756662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61474619-49c6-432c-bb36-68918ba7c271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:41.758168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61474619-49c6-432c-bb36-68918ba7c271 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:41.758550 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61474619-49c6-432c-bb36-68918ba7c271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:41.758899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61474619-49c6-432c-bb36-68918ba7c271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:41.760143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61474619-49c6-432c-bb36-68918ba7c271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:41.762799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61474619-49c6-432c-bb36-68918ba7c271 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:41.788034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1089/2165] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:46:41.791547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5f5a-f7a3-4077-9985-4fca44c337b2 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:41.806530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5f5a-f7a3-4077-9985-4fca44c337b2 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:41.814031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b11f0-b4ac-4f25-95a6-1fe19b7b3ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:41.815951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b11f0-b4ac-4f25-95a6-1fe19b7b3ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:41.816331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b11f0-b4ac-4f25-95a6-1fe19b7b3ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:41.816735 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b11f0-b4ac-4f25-95a6-1fe19b7b3ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:41.817514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b11f0-b4ac-4f25-95a6-1fe19b7b3ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:41.819577 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b11f0-b4ac-4f25-95a6-1fe19b7b3ba1 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:41.834732 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1077/2166] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:41.837874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61474619-49c6-432c-bb36-68918ba7c271 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:41.852145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61474619-49c6-432c-bb36-68918ba7c271 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:41.860642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aaa940-2592-4f67-b9b7-6ad8018f9da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:41.861666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aaa940-2592-4f67-b9b7-6ad8018f9da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:41.862103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aaa940-2592-4f67-b9b7-6ad8018f9da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:41.862476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aaa940-2592-4f67-b9b7-6ad8018f9da7 admin admin] PATH_INFO: `/v3/projects/74d60bb7f6f4463b84befafb64ce69a7/users/b49bef1b9e104042974e482a90abe58b/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:41.894244 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aaa940-2592-4f67-b9b7-6ad8018f9da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d60bb7f6f4463b84befafb64ce69a7, user_id=b49bef1b9e104042974e482a90abe58b, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:41.897704 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aaa940-2592-4f67-b9b7-6ad8018f9da7 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:41.921992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1090/2167] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:41 2022] PUT /identity/v3/projects/74d60bb7f6f4463b84befafb64ce69a7/users/b49bef1b9e104042974e482a90abe58b/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:46:41.934703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b510-c513-4df7-bc06-f1781ac165d2 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:41.935318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b510-c513-4df7-bc06-f1781ac165d2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:41.937751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b510-c513-4df7-bc06-f1781ac165d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:41.983270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4da3b510-c513-4df7-bc06-f1781ac165d2 None None] MFA Rules not processed for user `b49bef1b9e104042974e482a90abe58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:41.984978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4da3b510-c513-4df7-bc06-f1781ac165d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:42.064311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1078/2168] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:42.069388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aaa940-2592-4f67-b9b7-6ad8018f9da7 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:42.078164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:42.223109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aaa940-2592-4f67-b9b7-6ad8018f9da7 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:42.230940 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060c836-f013-4fa8-8c62-0aa78a5307af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:42.237071 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060c836-f013-4fa8-8c62-0aa78a5307af admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:42.237459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060c836-f013-4fa8-8c62-0aa78a5307af admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:42.238056 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060c836-f013-4fa8-8c62-0aa78a5307af admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:42.239930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060c836-f013-4fa8-8c62-0aa78a5307af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:42.242169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060c836-f013-4fa8-8c62-0aa78a5307af admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:42.304218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1091/2169] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:42 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:46:42.308118 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b510-c513-4df7-bc06-f1781ac165d2 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:42.324557 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b510-c513-4df7-bc06-f1781ac165d2 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:42.403427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394a36f-2cc4-446b-b6b0-10bb0dfd601b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:42.404248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394a36f-2cc4-446b-b6b0-10bb0dfd601b admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:42.404529 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394a36f-2cc4-446b-b6b0-10bb0dfd601b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:42.404998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394a36f-2cc4-446b-b6b0-10bb0dfd601b admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:42.405667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394a36f-2cc4-446b-b6b0-10bb0dfd601b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:42.407822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394a36f-2cc4-446b-b6b0-10bb0dfd601b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:42.443005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1079/2170] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:42 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:42.446818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060c836-f013-4fa8-8c62-0aa78a5307af admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:42.462077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060c836-f013-4fa8-8c62-0aa78a5307af admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:42.467984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21091f0f-25c0-4418-8bc3-0c462c12835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:42.468751 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21091f0f-25c0-4418-8bc3-0c462c12835f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:42.469184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21091f0f-25c0-4418-8bc3-0c462c12835f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:42.469587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21091f0f-25c0-4418-8bc3-0c462c12835f admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:42.470353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21091f0f-25c0-4418-8bc3-0c462c12835f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:42.472296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21091f0f-25c0-4418-8bc3-0c462c12835f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:42.486294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1092/2171] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:42.491722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394a36f-2cc4-446b-b6b0-10bb0dfd601b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:42.505990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394a36f-2cc4-446b-b6b0-10bb0dfd601b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:42.515020 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4685c-47b5-4155-a5f5-05d23d8cd73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:42.515711 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4685c-47b5-4155-a5f5-05d23d8cd73b admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:42.515989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4685c-47b5-4155-a5f5-05d23d8cd73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:42.516266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4685c-47b5-4155-a5f5-05d23d8cd73b admin admin] PATH_INFO: `/v3/projects/daa0faf3de8e4a908cd080a0f6b4428f/users/1c5b3cd973da442f985aa0a58633d03e/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:42.534220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:42.539767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4685c-47b5-4155-a5f5-05d23d8cd73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa0faf3de8e4a908cd080a0f6b4428f, user_id=1c5b3cd973da442f985aa0a58633d03e, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:42.543668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4685c-47b5-4155-a5f5-05d23d8cd73b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:42.566695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1080/2172] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:42 2022] PUT /identity/v3/projects/daa0faf3de8e4a908cd080a0f6b4428f/users/1c5b3cd973da442f985aa0a58633d03e/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:42.567443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21091f0f-25c0-4418-8bc3-0c462c12835f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:42.578009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21091f0f-25c0-4418-8bc3-0c462c12835f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:42.673609 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ffb74-28f8-4de5-8a73-a4dd00d20538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:42.674425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ffb74-28f8-4de5-8a73-a4dd00d20538 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:42.674926 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ffb74-28f8-4de5-8a73-a4dd00d20538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:42.675283 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ffb74-28f8-4de5-8a73-a4dd00d20538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:42.678181 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ffb74-28f8-4de5-8a73-a4dd00d20538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:42.681014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ffb74-28f8-4de5-8a73-a4dd00d20538 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:42.695083 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1093/2173] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:46:42.698311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4685c-47b5-4155-a5f5-05d23d8cd73b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:42.714095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4685c-47b5-4155-a5f5-05d23d8cd73b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:42.723923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c2255-023f-41ae-8faf-85b305d65768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:42.724622 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c2255-023f-41ae-8faf-85b305d65768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:42.725603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c2255-023f-41ae-8faf-85b305d65768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:42.725899 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c2255-023f-41ae-8faf-85b305d65768 admin admin] PATH_INFO: `/v3/projects/daa0faf3de8e4a908cd080a0f6b4428f/users/1c5b3cd973da442f985aa0a58633d03e/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:42.732607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c2255-023f-41ae-8faf-85b305d65768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa0faf3de8e4a908cd080a0f6b4428f, user_id=1c5b3cd973da442f985aa0a58633d03e, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:42.735306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c2255-023f-41ae-8faf-85b305d65768 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:42.786528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1081/2174] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:42 2022] PUT /identity/v3/projects/daa0faf3de8e4a908cd080a0f6b4428f/users/1c5b3cd973da442f985aa0a58633d03e/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:42.844578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36dcd3-79b7-48ec-bca5-2d3e89237498 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:42.845489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36dcd3-79b7-48ec-bca5-2d3e89237498 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:42.845932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36dcd3-79b7-48ec-bca5-2d3e89237498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:42.901918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-2c36dcd3-79b7-48ec-bca5-2d3e89237498 None None] MFA Rules not processed for user `1c5b3cd973da442f985aa0a58633d03e`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:42.903266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2c36dcd3-79b7-48ec-bca5-2d3e89237498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:43.020142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:43.028285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1094/2175] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:43.039695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c2255-023f-41ae-8faf-85b305d65768 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:43.043213 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ea5c2255-023f-41ae-8faf-85b305d65768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:43.183096 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c2255-023f-41ae-8faf-85b305d65768 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:43.196685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf93051-1971-4973-8be4-84a39a91733c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:43.197363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf93051-1971-4973-8be4-84a39a91733c service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:43.197646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf93051-1971-4973-8be4-84a39a91733c service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:43.197922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf93051-1971-4973-8be4-84a39a91733c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:43.219715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf93051-1971-4973-8be4-84a39a91733c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:43.222166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf93051-1971-4973-8be4-84a39a91733c service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:43.242627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1082/2176] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:43.406066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36dcd3-79b7-48ec-bca5-2d3e89237498 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:43.407520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2c36dcd3-79b7-48ec-bca5-2d3e89237498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:43.513319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf93051-1971-4973-8be4-84a39a91733c service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:43.514811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7bf93051-1971-4973-8be4-84a39a91733c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:43.573787 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36dcd3-79b7-48ec-bca5-2d3e89237498 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:43.588803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d580ae3-1504-466c-8d1e-8ef70c11ae6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:43.593271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580ae3-1504-466c-8d1e-8ef70c11ae6f service placement] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:43.593271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580ae3-1504-466c-8d1e-8ef70c11ae6f service placement] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:43.593271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580ae3-1504-466c-8d1e-8ef70c11ae6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:43.597087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0d580ae3-1504-466c-8d1e-8ef70c11ae6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:43.637785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d580ae3-1504-466c-8d1e-8ef70c11ae6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:43.639267 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d580ae3-1504-466c-8d1e-8ef70c11ae6f service placement] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:43.657452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1095/2177] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:43.699473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf93051-1971-4973-8be4-84a39a91733c service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:43.706339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:43.708014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:43.708279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:43.708546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] PATH_INFO: `/v3/users/bddd3d2bb8f647f2ba616141eb16622e` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:43.709966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bddd3d2bb8f647f2ba616141eb16622e)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:43.712191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:43.811660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bddd3d2bb8f647f2ba616141eb16622e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:43.821166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bddd3d2bb8f647f2ba616141eb16622e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:43.821897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bddd3d2bb8f647f2ba616141eb16622e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:43.841927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bddd3d2bb8f647f2ba616141eb16622e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:43.851245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bddd3d2bb8f647f2ba616141eb16622e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:43.858747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:43.868469 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1083/2178] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:43 2022] DELETE /identity/v3/users/bddd3d2bb8f647f2ba616141eb16622e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:46:43.890436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d580ae3-1504-466c-8d1e-8ef70c11ae6f service placement] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:43.906616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d580ae3-1504-466c-8d1e-8ef70c11ae6f service placement] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:44.038993 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:44.040527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.041006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.041450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] PATH_INFO: `/v3/users/b49bef1b9e104042974e482a90abe58b` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.042799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49bef1b9e104042974e482a90abe58b)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:44.045043 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:44.091415 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.093000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:44.121634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87d0e9-a645-4f8e-bf2f-d3c4c5090391 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:44.126119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49bef1b9e104042974e482a90abe58b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.129630 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:44.130306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.130581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.130855 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] PATH_INFO: `/v3/projects/ffc96760658d4e19af26d6cf22e94172` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.139475 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc96760658d4e19af26d6cf22e94172)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:44.141431 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:44.163268 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49bef1b9e104042974e482a90abe58b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.165224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49bef1b9e104042974e482a90abe58b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.178706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49bef1b9e104042974e482a90abe58b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.179706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-66b7e9d1-5b2b-499a-aaef-f6b851ed539f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49bef1b9e104042974e482a90abe58b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.209582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1096/2179] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:43 2022] DELETE /identity/v3/users/b49bef1b9e104042974e482a90abe58b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:46:44.214612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa400d49-7314-46b7-8cc2-c2efba9f41a3 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.214894 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa400d49-7314-46b7-8cc2-c2efba9f41a3 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.215408 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa400d49-7314-46b7-8cc2-c2efba9f41a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.226938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc96760658d4e19af26d6cf22e94172', 'request_id': 'req-eee6e183-ec00-427e-b9c5-1e3090d3c952'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.235627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc96760658d4e19af26d6cf22e94172', 'request_id': 'req-eee6e183-ec00-427e-b9c5-1e3090d3c952'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.244534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-fa400d49-7314-46b7-8cc2-c2efba9f41a3 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:44.246969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc96760658d4e19af26d6cf22e94172', 'request_id': 'req-eee6e183-ec00-427e-b9c5-1e3090d3c952'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.255576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-fa400d49-7314-46b7-8cc2-c2efba9f41a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:44.264630 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:44.268005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] The token cache is being invalidate because project ffc96760658d4e19af26d6cf22e94172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:44.268302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.268980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-eee6e183-ec00-427e-b9c5-1e3090d3c952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.275112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1084/2180] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:44 2022] DELETE /identity/v3/projects/ffc96760658d4e19af26d6cf22e94172 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:44.282116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06efe9-79fd-4917-8772-2b65609af897 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.282660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06efe9-79fd-4917-8772-2b65609af897 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.282660 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06efe9-79fd-4917-8772-2b65609af897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.309227 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ef06efe9-79fd-4917-8772-2b65609af897 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:44.310730 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ef06efe9-79fd-4917-8772-2b65609af897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:44.343523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1097/2181] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:44.353257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa400d49-7314-46b7-8cc2-c2efba9f41a3 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.354631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-fa400d49-7314-46b7-8cc2-c2efba9f41a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:44.398644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1085/2182] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:44.406390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e3065-1fea-475c-bcb0-781f579867a5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.406798 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e3065-1fea-475c-bcb0-781f579867a5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.407166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e3065-1fea-475c-bcb0-781f579867a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.428833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa400d49-7314-46b7-8cc2-c2efba9f41a3 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:44.429949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-b08e3065-1fea-475c-bcb0-781f579867a5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:44.431343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b08e3065-1fea-475c-bcb0-781f579867a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:44.432461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87c8a4-f190-49e3-83e8-f77c942b3c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:44.433097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87c8a4-f190-49e3-83e8-f77c942b3c75 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.433377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87c8a4-f190-49e3-83e8-f77c942b3c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.433643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87c8a4-f190-49e3-83e8-f77c942b3c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.441997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1086/2183] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:46:44.444103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87c8a4-f190-49e3-83e8-f77c942b3c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:44.445697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87c8a4-f190-49e3-83e8-f77c942b3c75 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:44.446085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e3065-1fea-475c-bcb0-781f579867a5 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.455535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e3065-1fea-475c-bcb0-781f579867a5 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:44.458177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1098/2184] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:46:44.461270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1447c-d057-480a-be4e-e57904e1c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:44.461871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1447c-d057-480a-be4e-e57904e1c97c admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.462142 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1447c-d057-480a-be4e-e57904e1c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.462409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1447c-d057-480a-be4e-e57904e1c97c admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.463056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1447c-d057-480a-be4e-e57904e1c97c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:46:44.463350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1447c-d057-480a-be4e-e57904e1c97c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:44.464967 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1447c-d057-480a-be4e-e57904e1c97c admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:44.476885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1087/2185] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:46:44.481173 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87c8a4-f190-49e3-83e8-f77c942b3c75 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.492383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87c8a4-f190-49e3-83e8-f77c942b3c75 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:44.497258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d25cc7-1a76-4589-92f0-2bddc5fb6c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:44.497889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25cc7-1a76-4589-92f0-2bddc5fb6c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.498184 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25cc7-1a76-4589-92f0-2bddc5fb6c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.498459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25cc7-1a76-4589-92f0-2bddc5fb6c06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.499106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d25cc7-1a76-4589-92f0-2bddc5fb6c06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:44.501025 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d25cc7-1a76-4589-92f0-2bddc5fb6c06 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:44.536057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1099/2186] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:44 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:44.540528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1447c-d057-480a-be4e-e57904e1c97c admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.551848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1447c-d057-480a-be4e-e57904e1c97c admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:44.630851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e121a2-1a60-46af-8b51-4577daa3456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:44.631467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e121a2-1a60-46af-8b51-4577daa3456b admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.631820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e121a2-1a60-46af-8b51-4577daa3456b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.632166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e121a2-1a60-46af-8b51-4577daa3456b admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.633275 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e121a2-1a60-46af-8b51-4577daa3456b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:44.635049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e121a2-1a60-46af-8b51-4577daa3456b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:44.646823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:44.665364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1088/2187] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:44 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:44.669516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d25cc7-1a76-4589-92f0-2bddc5fb6c06 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.680752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d25cc7-1a76-4589-92f0-2bddc5fb6c06 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:44.685742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da35ae-f0c0-4977-8ef4-5104416b461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:44.686472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da35ae-f0c0-4977-8ef4-5104416b461e admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.686836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da35ae-f0c0-4977-8ef4-5104416b461e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.687174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da35ae-f0c0-4977-8ef4-5104416b461e admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.687987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da35ae-f0c0-4977-8ef4-5104416b461e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:44.689629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da35ae-f0c0-4977-8ef4-5104416b461e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:44.704775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1100/2188] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:46:44.706044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e121a2-1a60-46af-8b51-4577daa3456b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.718604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e121a2-1a60-46af-8b51-4577daa3456b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:44.724549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30ffb-68a0-4406-9139-faaef89636ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:44.725323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30ffb-68a0-4406-9139-faaef89636ae admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.725788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30ffb-68a0-4406-9139-faaef89636ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.726154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30ffb-68a0-4406-9139-faaef89636ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.726895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc30ffb-68a0-4406-9139-faaef89636ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:44.729270 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc30ffb-68a0-4406-9139-faaef89636ae admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:44.741881 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1089/2189] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:44.746600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da35ae-f0c0-4977-8ef4-5104416b461e admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.757270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da35ae-f0c0-4977-8ef4-5104416b461e admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:44.762989 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57841ebe-f27c-4e69-a31e-1b0a9c83dc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:44.765576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57841ebe-f27c-4e69-a31e-1b0a9c83dc14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.765576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57841ebe-f27c-4e69-a31e-1b0a9c83dc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.765576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57841ebe-f27c-4e69-a31e-1b0a9c83dc14 admin admin] PATH_INFO: `/v3/projects/11e79ee67bfb4ee2bf5c9e855cbf5b1f/users/18b1cd00c6b246d7b9214d51f2899040/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.782252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57841ebe-f27c-4e69-a31e-1b0a9c83dc14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e79ee67bfb4ee2bf5c9e855cbf5b1f, user_id=18b1cd00c6b246d7b9214d51f2899040, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:44.787920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57841ebe-f27c-4e69-a31e-1b0a9c83dc14 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:44.789157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30ffb-68a0-4406-9139-faaef89636ae admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.795277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0fc30ffb-68a0-4406-9139-faaef89636ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:44.805477 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1101/2190] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:44 2022] PUT /identity/v3/projects/11e79ee67bfb4ee2bf5c9e855cbf5b1f/users/18b1cd00c6b246d7b9214d51f2899040/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:44.823272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30ffb-68a0-4406-9139-faaef89636ae admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:44.826606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122d4e2-a81b-4e2e-97ac-2fcc1c825da0 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.827953 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122d4e2-a81b-4e2e-97ac-2fcc1c825da0 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.828618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122d4e2-a81b-4e2e-97ac-2fcc1c825da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.864422 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-3122d4e2-a81b-4e2e-97ac-2fcc1c825da0 None None] MFA Rules not processed for user `18b1cd00c6b246d7b9214d51f2899040`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:44.868117 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3122d4e2-a81b-4e2e-97ac-2fcc1c825da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:44.890352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:44.891152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:44.891451 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:44.891794 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] PATH_INFO: `/v3/projects/74d60bb7f6f4463b84befafb64ce69a7` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:44.893467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d60bb7f6f4463b84befafb64ce69a7)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:44.895956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:44.918376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:44.944255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1102/2191] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:44.956541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122d4e2-a81b-4e2e-97ac-2fcc1c825da0 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.958519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3122d4e2-a81b-4e2e-97ac-2fcc1c825da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:44.959645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d60bb7f6f4463b84befafb64ce69a7', 'request_id': 'req-b137b275-14fa-4f94-9026-37aaab20688b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.967311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d60bb7f6f4463b84befafb64ce69a7', 'request_id': 'req-b137b275-14fa-4f94-9026-37aaab20688b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.976739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d60bb7f6f4463b84befafb64ce69a7', 'request_id': 'req-b137b275-14fa-4f94-9026-37aaab20688b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.990064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] The token cache is being invalidate because project 74d60bb7f6f4463b84befafb64ce69a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:44.990354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.991009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:44.992513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1090/2192] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:44 2022] DELETE /identity/v3/projects/74d60bb7f6f4463b84befafb64ce69a7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:44.996339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:44.997700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:45.027016 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122d4e2-a81b-4e2e-97ac-2fcc1c825da0 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:45.062478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137b275-14fa-4f94-9026-37aaab20688b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:45.068652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:45.069302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:45.069579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:45.069848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] PATH_INFO: `/v3/users/1c5b3cd973da442f985aa0a58633d03e` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:45.070973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5b3cd973da442f985aa0a58633d03e)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:45.073241 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:45.075705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182e1d-e798-409c-932f-168681866b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:45.076302 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182e1d-e798-409c-932f-168681866b78 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:45.076583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182e1d-e798-409c-932f-168681866b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:45.076872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182e1d-e798-409c-932f-168681866b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:45.078644 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-18182e1d-e798-409c-932f-168681866b78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:45.087886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:45.121066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5b3cd973da442f985aa0a58633d03e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:45.130090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5b3cd973da442f985aa0a58633d03e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:45.130805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5b3cd973da442f985aa0a58633d03e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:45.142592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18182e1d-e798-409c-932f-168681866b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:45.143438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5b3cd973da442f985aa0a58633d03e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:45.144432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18182e1d-e798-409c-932f-168681866b78 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:45.154322 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-94900421-a801-4d3b-92bf-e89264381fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5b3cd973da442f985aa0a58633d03e'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:45.155358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1103/2193] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:46:45.162605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1091/2194] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:44 2022] DELETE /identity/v3/users/1c5b3cd973da442f985aa0a58633d03e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:46:45.583373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182e1d-e798-409c-932f-168681866b78 service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:45.595930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182e1d-e798-409c-932f-168681866b78 service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:45.661552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:45.669271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:45.669992 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:45.670339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:45.670713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] PATH_INFO: `/v3/projects/daa0faf3de8e4a908cd080a0f6b4428f` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:45.671934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa0faf3de8e4a908cd080a0f6b4428f)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:45.674018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:45.720107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa0faf3de8e4a908cd080a0f6b4428f', 'request_id': 'req-67914651-398d-49a7-8948-c5ba8a2b9286'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:45.726170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa0faf3de8e4a908cd080a0f6b4428f', 'request_id': 'req-67914651-398d-49a7-8948-c5ba8a2b9286'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:45.733512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa0faf3de8e4a908cd080a0f6b4428f', 'request_id': 'req-67914651-398d-49a7-8948-c5ba8a2b9286'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:45.745457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] The token cache is being invalidate because project daa0faf3de8e4a908cd080a0f6b4428f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:45.745744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:45.746382 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-67914651-398d-49a7-8948-c5ba8a2b9286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:45.748120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1104/2195] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:45 2022] DELETE /identity/v3/projects/daa0faf3de8e4a908cd080a0f6b4428f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:45.755572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a5d11-6d5f-4684-82bd-d4b5aaf1f378 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:45.755572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a5d11-6d5f-4684-82bd-d4b5aaf1f378 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:45.755572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a5d11-6d5f-4684-82bd-d4b5aaf1f378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:45.775727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-166a5d11-6d5f-4684-82bd-d4b5aaf1f378 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:45.777214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-166a5d11-6d5f-4684-82bd-d4b5aaf1f378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:45.840806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1092/2196] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:45.849205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c0789-917b-4a53-88fc-ab35905dc6c0 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:45.849484 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c0789-917b-4a53-88fc-ab35905dc6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:45.849781 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c0789-917b-4a53-88fc-ab35905dc6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:45.870295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-023c0789-917b-4a53-88fc-ab35905dc6c0 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:45.871597 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-023c0789-917b-4a53-88fc-ab35905dc6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:45.879465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1105/2197] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:45.883752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a5d11-6d5f-4684-82bd-d4b5aaf1f378 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:45.893980 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a5d11-6d5f-4684-82bd-d4b5aaf1f378 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:45.898651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1290030-53e8-44d3-bc74-2a13b189b0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:45.899260 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1290030-53e8-44d3-bc74-2a13b189b0cc admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:45.899539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1290030-53e8-44d3-bc74-2a13b189b0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:45.899811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1290030-53e8-44d3-bc74-2a13b189b0cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:45.900459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1290030-53e8-44d3-bc74-2a13b189b0cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:46:45.900763 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1290030-53e8-44d3-bc74-2a13b189b0cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:45.902397 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1290030-53e8-44d3-bc74-2a13b189b0cc admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:45.917831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1093/2198] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:46:45.918934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c0789-917b-4a53-88fc-ab35905dc6c0 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:45.928689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c0789-917b-4a53-88fc-ab35905dc6c0 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:45.933796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a8e97-2451-4a68-8f5a-76fdf4b42eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:45.934379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a8e97-2451-4a68-8f5a-76fdf4b42eab admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:45.934659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a8e97-2451-4a68-8f5a-76fdf4b42eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:45.934931 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a8e97-2451-4a68-8f5a-76fdf4b42eab admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:45.935562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a8e97-2451-4a68-8f5a-76fdf4b42eab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:45.937512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a8e97-2451-4a68-8f5a-76fdf4b42eab admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:45.965867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1106/2199] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:45 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:46:45.970027 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1290030-53e8-44d3-bc74-2a13b189b0cc admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:45.980568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1290030-53e8-44d3-bc74-2a13b189b0cc admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:46.037867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f67c8-db94-4e6b-bb1b-7a8a227b26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:46.038460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f67c8-db94-4e6b-bb1b-7a8a227b26c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:46.038739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f67c8-db94-4e6b-bb1b-7a8a227b26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:46.039008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f67c8-db94-4e6b-bb1b-7a8a227b26c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:46.039625 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f67c8-db94-4e6b-bb1b-7a8a227b26c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:46.041567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f67c8-db94-4e6b-bb1b-7a8a227b26c2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:46.067214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1094/2200] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:45 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:46.068528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a8e97-2451-4a68-8f5a-76fdf4b42eab admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:46.078754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a8e97-2451-4a68-8f5a-76fdf4b42eab admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:46.084237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dbea1-3605-4afa-8277-71b913b1d673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:46.084861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dbea1-3605-4afa-8277-71b913b1d673 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:46.085156 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dbea1-3605-4afa-8277-71b913b1d673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:46.085427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dbea1-3605-4afa-8277-71b913b1d673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:46.086072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dbea1-3605-4afa-8277-71b913b1d673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:46.087662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dbea1-3605-4afa-8277-71b913b1d673 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:46.098397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1107/2201] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:46:46.102289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f67c8-db94-4e6b-bb1b-7a8a227b26c2 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:46.112171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f67c8-db94-4e6b-bb1b-7a8a227b26c2 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:46.116898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd360c64-39e8-4d5a-8e09-b91f0cbc66ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:46.117501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd360c64-39e8-4d5a-8e09-b91f0cbc66ce admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:46.117778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd360c64-39e8-4d5a-8e09-b91f0cbc66ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:46.118049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd360c64-39e8-4d5a-8e09-b91f0cbc66ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:46.118733 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd360c64-39e8-4d5a-8e09-b91f0cbc66ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:46.120297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd360c64-39e8-4d5a-8e09-b91f0cbc66ce admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:46.122428 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:46.132796 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1095/2202] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:46.136476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dbea1-3605-4afa-8277-71b913b1d673 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:46.147195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dbea1-3605-4afa-8277-71b913b1d673 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:46.151949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fa02f-5c7f-4a0d-8bd5-6dc7e1cee5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:46.152592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fa02f-5c7f-4a0d-8bd5-6dc7e1cee5c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:46.152892 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fa02f-5c7f-4a0d-8bd5-6dc7e1cee5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:46.153281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fa02f-5c7f-4a0d-8bd5-6dc7e1cee5c4 admin admin] PATH_INFO: `/v3/projects/725aafacf05848b4910c68e88cbdc10a/users/463c027b301d4382b650adf7ae6e8732/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:46.172076 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fa02f-5c7f-4a0d-8bd5-6dc7e1cee5c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=725aafacf05848b4910c68e88cbdc10a, user_id=463c027b301d4382b650adf7ae6e8732, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:46.174916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fa02f-5c7f-4a0d-8bd5-6dc7e1cee5c4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:46.188734 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1108/2203] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:46 2022] PUT /identity/v3/projects/725aafacf05848b4910c68e88cbdc10a/users/463c027b301d4382b650adf7ae6e8732/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:46.200304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c38e1f-8a49-42e1-8936-d0804bf9c238 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:46.200591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c38e1f-8a49-42e1-8936-d0804bf9c238 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:46.200904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c38e1f-8a49-42e1-8936-d0804bf9c238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:46.230805 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-29c38e1f-8a49-42e1-8936-d0804bf9c238 None None] MFA Rules not processed for user `463c027b301d4382b650adf7ae6e8732`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:46.232015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-29c38e1f-8a49-42e1-8936-d0804bf9c238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:46.294038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1096/2204] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:46 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:46.305587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fa02f-5c7f-4a0d-8bd5-6dc7e1cee5c4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:46.307298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-780fa02f-5c7f-4a0d-8bd5-6dc7e1cee5c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:46.363627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fa02f-5c7f-4a0d-8bd5-6dc7e1cee5c4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:46.367179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e1822-df17-47fa-943c-49bf6859e3ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:46.367771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e1822-df17-47fa-943c-49bf6859e3ed service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:46.368045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e1822-df17-47fa-943c-49bf6859e3ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:46.368311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e1822-df17-47fa-943c-49bf6859e3ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:46.376341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e1822-df17-47fa-943c-49bf6859e3ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:46.377779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e1822-df17-47fa-943c-49bf6859e3ed service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:46.386906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1109/2205] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:51.068140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c38e1f-8a49-42e1-8936-d0804bf9c238 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:51.078013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c38e1f-8a49-42e1-8936-d0804bf9c238 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:51.082453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77b126-41ee-4abe-ad41-d650fc4025ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:51.083045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77b126-41ee-4abe-ad41-d650fc4025ba service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:51.083346 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77b126-41ee-4abe-ad41-d650fc4025ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:51.083613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77b126-41ee-4abe-ad41-d650fc4025ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:51.085359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6a77b126-41ee-4abe-ad41-d650fc4025ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:51.147974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77b126-41ee-4abe-ad41-d650fc4025ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:51.149382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77b126-41ee-4abe-ad41-d650fc4025ba service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:51.158331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1097/2206] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:53.415140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa05ef-2f65-4f4a-8713-eccf472d9c5b None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:53.415605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa05ef-2f65-4f4a-8713-eccf472d9c5b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:53.415711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa05ef-2f65-4f4a-8713-eccf472d9c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:53.435494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-22fa05ef-2f65-4f4a-8713-eccf472d9c5b None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:53.436691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-22fa05ef-2f65-4f4a-8713-eccf472d9c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:53.454930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:53.493450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1110/2207] 10.222.0.17 () {58 vars in 1009 bytes} [Tue Oct 18 11:46:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:53.505331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77b126-41ee-4abe-ad41-d650fc4025ba service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:53.506727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6a77b126-41ee-4abe-ad41-d650fc4025ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:53.573154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77b126-41ee-4abe-ad41-d650fc4025ba service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:53.578116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c921cf1-3c79-4d22-bcf4-bf6edfe7ad34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:53.578975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c921cf1-3c79-4d22-bcf4-bf6edfe7ad34 service swift] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:53.579246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c921cf1-3c79-4d22-bcf4-bf6edfe7ad34 service swift] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:53.579507 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c921cf1-3c79-4d22-bcf4-bf6edfe7ad34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:53.588152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c921cf1-3c79-4d22-bcf4-bf6edfe7ad34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:53.589740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c921cf1-3c79-4d22-bcf4-bf6edfe7ad34 service swift] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:53.599382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1098/2208] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:46:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 18 11:46:53.633039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa05ef-2f65-4f4a-8713-eccf472d9c5b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:53.720929 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa05ef-2f65-4f4a-8713-eccf472d9c5b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:53.740999 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:53.740999 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:53.740999 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:53.740999 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] PATH_INFO: `/v3/users/463c027b301d4382b650adf7ae6e8732` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:53.741601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=463c027b301d4382b650adf7ae6e8732)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:53.746768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:53.825866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '463c027b301d4382b650adf7ae6e8732'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:53.856664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '463c027b301d4382b650adf7ae6e8732'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:53.857261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '463c027b301d4382b650adf7ae6e8732'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:53.893470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '463c027b301d4382b650adf7ae6e8732'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:53.895126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '463c027b301d4382b650adf7ae6e8732'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:53.931030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1111/2209] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:53 2022] DELETE /identity/v3/users/463c027b301d4382b650adf7ae6e8732 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:46:53.934344 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52178d8a-0085-4e76-be0d-8ecf11879cf4 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:53.934693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52178d8a-0085-4e76-be0d-8ecf11879cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:53.934966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52178d8a-0085-4e76-be0d-8ecf11879cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:53.995239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-52178d8a-0085-4e76-be0d-8ecf11879cf4 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:53.997938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-52178d8a-0085-4e76-be0d-8ecf11879cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:54.014728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:54.105107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1099/2210] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:54.112049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:54.113561 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:54.192162 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:54.197616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59e788-0760-4a4f-a598-4632ef42afd1 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:54.201470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2cfa6-4a1a-4c16-982c-7100f75befa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:54.203518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2cfa6-4a1a-4c16-982c-7100f75befa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:54.204123 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2cfa6-4a1a-4c16-982c-7100f75befa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:54.204419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2cfa6-4a1a-4c16-982c-7100f75befa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:54.217621 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2cfa6-4a1a-4c16-982c-7100f75befa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:54.219075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2cfa6-4a1a-4c16-982c-7100f75befa9 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:54.230828 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1112/2211] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:54.570291 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52178d8a-0085-4e76-be0d-8ecf11879cf4 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:54.571770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-52178d8a-0085-4e76-be0d-8ecf11879cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:54.589210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52178d8a-0085-4e76-be0d-8ecf11879cf4 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:54.602718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:54.603580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:54.604006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:54.604447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] PATH_INFO: `/v3/projects/725aafacf05848b4910c68e88cbdc10a` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:54.606394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] RBAC: Authorizing `identity:delete_project(project_id=725aafacf05848b4910c68e88cbdc10a)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:54.608485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:54.664060 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725aafacf05848b4910c68e88cbdc10a', 'request_id': 'req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:54.669695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725aafacf05848b4910c68e88cbdc10a', 'request_id': 'req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:54.677908 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725aafacf05848b4910c68e88cbdc10a', 'request_id': 'req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:54.690222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] The token cache is being invalidate because project 725aafacf05848b4910c68e88cbdc10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:54.690632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:54.691325 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71e2bb02-2a51-495a-a10c-fa4e2d9eb27a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:54.692828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1100/2212] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:54 2022] DELETE /identity/v3/projects/725aafacf05848b4910c68e88cbdc10a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:55.953913 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2cfa6-4a1a-4c16-982c-7100f75befa9 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:55.955419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4de2cfa6-4a1a-4c16-982c-7100f75befa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:55.968050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2cfa6-4a1a-4c16-982c-7100f75befa9 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:56.023810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:56.024438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:56.024722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:56.025045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] PATH_INFO: `/v3/users/18b1cd00c6b246d7b9214d51f2899040` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:56.026167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18b1cd00c6b246d7b9214d51f2899040)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:56.028075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:56.078414 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18b1cd00c6b246d7b9214d51f2899040'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:56.094846 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18b1cd00c6b246d7b9214d51f2899040'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:56.095791 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18b1cd00c6b246d7b9214d51f2899040'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:56.109055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18b1cd00c6b246d7b9214d51f2899040'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:56.110164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18b1cd00c6b246d7b9214d51f2899040'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:56.129235 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1113/2213] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:46:55 2022] DELETE /identity/v3/users/18b1cd00c6b246d7b9214d51f2899040 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:56.135655 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212fa18-231a-45ce-b13e-07756bddfcbe None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:56.136619 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212fa18-231a-45ce-b13e-07756bddfcbe None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:56.139169 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212fa18-231a-45ce-b13e-07756bddfcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:56.166994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-c212fa18-231a-45ce-b13e-07756bddfcbe None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:56.168346 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c212fa18-231a-45ce-b13e-07756bddfcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:56.233100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1101/2214] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:46:56.244473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:56.245983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:56.311706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f323dbf-f87d-4652-8720-ee7d0eb7f8b4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:56.315853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ef98c-afeb-431a-a1a0-147f224bb509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:56.316515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ef98c-afeb-431a-a1a0-147f224bb509 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:56.316860 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ef98c-afeb-431a-a1a0-147f224bb509 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:56.317133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ef98c-afeb-431a-a1a0-147f224bb509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:56.326852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ef98c-afeb-431a-a1a0-147f224bb509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:56.328785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ef98c-afeb-431a-a1a0-147f224bb509 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:56.340328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1114/2215] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:56.712470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212fa18-231a-45ce-b13e-07756bddfcbe None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:56.714182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c212fa18-231a-45ce-b13e-07756bddfcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:56.734000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212fa18-231a-45ce-b13e-07756bddfcbe None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:56.739473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:56.740645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:56.740645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:56.740921 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] PATH_INFO: `/v3/projects/11e79ee67bfb4ee2bf5c9e855cbf5b1f` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:56.742311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11e79ee67bfb4ee2bf5c9e855cbf5b1f)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:56.744526 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:56.788958 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:56.803624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11e79ee67bfb4ee2bf5c9e855cbf5b1f', 'request_id': 'req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:56.810960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11e79ee67bfb4ee2bf5c9e855cbf5b1f', 'request_id': 'req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:56.818236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11e79ee67bfb4ee2bf5c9e855cbf5b1f', 'request_id': 'req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:56.830508 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] The token cache is being invalidate because project 11e79ee67bfb4ee2bf5c9e855cbf5b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:46:56.831123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:56.831873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f963f8a9-1fd4-4e2c-99ba-24bc99aa12c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:46:56.834001 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1102/2216] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:46:56 2022] DELETE /identity/v3/projects/11e79ee67bfb4ee2bf5c9e855cbf5b1f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:56.839770 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc84a1-6d9c-49cd-9049-dbbe9cf3adbd None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:56.840582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc84a1-6d9c-49cd-9049-dbbe9cf3adbd None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:56.841010 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc84a1-6d9c-49cd-9049-dbbe9cf3adbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:56.863374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-eecc84a1-6d9c-49cd-9049-dbbe9cf3adbd None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:56.864617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-eecc84a1-6d9c-49cd-9049-dbbe9cf3adbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:56.875077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:56.931956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1115/2217] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:46:56.940477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e631df4-502c-42a6-a144-c6b7485bc629 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:56.940883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e631df4-502c-42a6-a144-c6b7485bc629 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:56.941166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e631df4-502c-42a6-a144-c6b7485bc629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:56.961183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-6e631df4-502c-42a6-a144-c6b7485bc629 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:56.962388 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6e631df4-502c-42a6-a144-c6b7485bc629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:56.970580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1103/2218] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:56.975459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc84a1-6d9c-49cd-9049-dbbe9cf3adbd None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:56.987452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc84a1-6d9c-49cd-9049-dbbe9cf3adbd None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:56.992657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121c500-cdb6-4758-bdc0-538302251445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:56.993348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121c500-cdb6-4758-bdc0-538302251445 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:56.993661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121c500-cdb6-4758-bdc0-538302251445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:56.993915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121c500-cdb6-4758-bdc0-538302251445 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:56.994558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121c500-cdb6-4758-bdc0-538302251445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:46:56.994856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121c500-cdb6-4758-bdc0-538302251445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:56.996473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121c500-cdb6-4758-bdc0-538302251445 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.008404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1116/2219] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:46:57.012576 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e631df4-502c-42a6-a144-c6b7485bc629 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.022070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e631df4-502c-42a6-a144-c6b7485bc629 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.029356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58e5dd-c1c4-4c07-a8a2-af1561f44250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.029997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58e5dd-c1c4-4c07-a8a2-af1561f44250 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.030254 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58e5dd-c1c4-4c07-a8a2-af1561f44250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.030549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58e5dd-c1c4-4c07-a8a2-af1561f44250 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.031166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58e5dd-c1c4-4c07-a8a2-af1561f44250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.033141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58e5dd-c1c4-4c07-a8a2-af1561f44250 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.065525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1104/2220] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:57 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:57.066769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121c500-cdb6-4758-bdc0-538302251445 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.077388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121c500-cdb6-4758-bdc0-538302251445 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.139232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab42481-fdf9-421e-bccb-3006bbf50948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.139851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab42481-fdf9-421e-bccb-3006bbf50948 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.140126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab42481-fdf9-421e-bccb-3006bbf50948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.140400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab42481-fdf9-421e-bccb-3006bbf50948 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.141078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab42481-fdf9-421e-bccb-3006bbf50948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.143000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab42481-fdf9-421e-bccb-3006bbf50948 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.174022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1117/2221] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:57 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:57.175852 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58e5dd-c1c4-4c07-a8a2-af1561f44250 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.190071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58e5dd-c1c4-4c07-a8a2-af1561f44250 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.195549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c239-85f7-4093-8976-a562f687bf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.197188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c239-85f7-4093-8976-a562f687bf93 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.197568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c239-85f7-4093-8976-a562f687bf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.197910 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c239-85f7-4093-8976-a562f687bf93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.198575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c239-85f7-4093-8976-a562f687bf93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.200362 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c239-85f7-4093-8976-a562f687bf93 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.211772 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1105/2222] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:57.215942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab42481-fdf9-421e-bccb-3006bbf50948 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.227492 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab42481-fdf9-421e-bccb-3006bbf50948 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.237298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16b308-6e27-467f-9c66-917ffce7f1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.238179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16b308-6e27-467f-9c66-917ffce7f1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.238459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16b308-6e27-467f-9c66-917ffce7f1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.238811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16b308-6e27-467f-9c66-917ffce7f1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.239599 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16b308-6e27-467f-9c66-917ffce7f1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.241779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16b308-6e27-467f-9c66-917ffce7f1b8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.271462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1118/2223] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:57.272827 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c239-85f7-4093-8976-a562f687bf93 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.283711 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c239-85f7-4093-8976-a562f687bf93 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.289105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5a71-d358-410c-ae09-3ae476b92b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.289768 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5a71-d358-410c-ae09-3ae476b92b23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.290047 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5a71-d358-410c-ae09-3ae476b92b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.290320 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5a71-d358-410c-ae09-3ae476b92b23 admin admin] PATH_INFO: `/v3/projects/304cf96f8c6a4993a1bf3122efb014a9/users/86d208c795024a729f1f95dd07c2e214/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.306907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa5a71-d358-410c-ae09-3ae476b92b23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=304cf96f8c6a4993a1bf3122efb014a9, user_id=86d208c795024a729f1f95dd07c2e214, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.310651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa5a71-d358-410c-ae09-3ae476b92b23 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.327655 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1106/2224] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:57 2022] PUT /identity/v3/projects/304cf96f8c6a4993a1bf3122efb014a9/users/86d208c795024a729f1f95dd07c2e214/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:57.340371 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b85fba-4f21-48b0-ab28-60dc408f2f32 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.340656 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b85fba-4f21-48b0-ab28-60dc408f2f32 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.340964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b85fba-4f21-48b0-ab28-60dc408f2f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.372230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-07b85fba-4f21-48b0-ab28-60dc408f2f32 None None] MFA Rules not processed for user `86d208c795024a729f1f95dd07c2e214`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:57.373633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-07b85fba-4f21-48b0-ab28-60dc408f2f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:57.443540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1119/2225] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:57.448074 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5a71-d358-410c-ae09-3ae476b92b23 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.520343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5a71-d358-410c-ae09-3ae476b92b23 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.525091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fba53c-cf92-4559-a08a-095b4f26cea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.525710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fba53c-cf92-4559-a08a-095b4f26cea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.526120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fba53c-cf92-4559-a08a-095b4f26cea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.526398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fba53c-cf92-4559-a08a-095b4f26cea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.527064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fba53c-cf92-4559-a08a-095b4f26cea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.528976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fba53c-cf92-4559-a08a-095b4f26cea6 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.559145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1107/2226] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:46:57 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:57.563203 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b85fba-4f21-48b0-ab28-60dc408f2f32 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.574681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b85fba-4f21-48b0-ab28-60dc408f2f32 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.579589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:57.634748 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8eeb1e-3132-4705-9539-855a8ccd01f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.635359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eeb1e-3132-4705-9539-855a8ccd01f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.635628 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eeb1e-3132-4705-9539-855a8ccd01f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.635906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eeb1e-3132-4705-9539-855a8ccd01f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.636539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8eeb1e-3132-4705-9539-855a8ccd01f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.638476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8eeb1e-3132-4705-9539-855a8ccd01f5 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.667772 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1120/2227] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:46:57 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:46:57.671877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fba53c-cf92-4559-a08a-095b4f26cea6 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.683296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fba53c-cf92-4559-a08a-095b4f26cea6 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.688939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b74442-64d3-4a1c-a442-6e3aa7830758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.689818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b74442-64d3-4a1c-a442-6e3aa7830758 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.690289 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b74442-64d3-4a1c-a442-6e3aa7830758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.690785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b74442-64d3-4a1c-a442-6e3aa7830758 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.692141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b74442-64d3-4a1c-a442-6e3aa7830758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.694264 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b74442-64d3-4a1c-a442-6e3aa7830758 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.710145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1108/2228] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:46:57.714383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8eeb1e-3132-4705-9539-855a8ccd01f5 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.725769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8eeb1e-3132-4705-9539-855a8ccd01f5 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.732302 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9eeaa-fe49-4732-b6da-c2c3acb47076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.733170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9eeaa-fe49-4732-b6da-c2c3acb47076 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.733458 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9eeaa-fe49-4732-b6da-c2c3acb47076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.733739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9eeaa-fe49-4732-b6da-c2c3acb47076 admin admin] PATH_INFO: `/v3/projects/5028a97c345b4a76baa7e8e67e3eab2d/users/441c4e2362af45a388d8f3e58e3f6582/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.750895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9eeaa-fe49-4732-b6da-c2c3acb47076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5028a97c345b4a76baa7e8e67e3eab2d, user_id=441c4e2362af45a388d8f3e58e3f6582, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.753715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9eeaa-fe49-4732-b6da-c2c3acb47076 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.772768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1121/2229] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:57 2022] PUT /identity/v3/projects/5028a97c345b4a76baa7e8e67e3eab2d/users/441c4e2362af45a388d8f3e58e3f6582/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:46:57.776863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b74442-64d3-4a1c-a442-6e3aa7830758 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.788436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b74442-64d3-4a1c-a442-6e3aa7830758 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.827841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:46:57.861233 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b8edb-d014-47c6-bc25-6e288321bff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.861854 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b8edb-d014-47c6-bc25-6e288321bff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.862132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b8edb-d014-47c6-bc25-6e288321bff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.862434 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b8edb-d014-47c6-bc25-6e288321bff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.863129 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b8edb-d014-47c6-bc25-6e288321bff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.864735 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b8edb-d014-47c6-bc25-6e288321bff7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.875055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1109/2230] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:46:57.879947 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9eeaa-fe49-4732-b6da-c2c3acb47076 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:57.890601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9eeaa-fe49-4732-b6da-c2c3acb47076 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:57.895973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476efc6-239a-487a-a0b8-2fe3c73caf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:57.896629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476efc6-239a-487a-a0b8-2fe3c73caf6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.896950 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476efc6-239a-487a-a0b8-2fe3c73caf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.897227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476efc6-239a-487a-a0b8-2fe3c73caf6c admin admin] PATH_INFO: `/v3/projects/5028a97c345b4a76baa7e8e67e3eab2d/users/441c4e2362af45a388d8f3e58e3f6582/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.899149 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476efc6-239a-487a-a0b8-2fe3c73caf6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5028a97c345b4a76baa7e8e67e3eab2d, user_id=441c4e2362af45a388d8f3e58e3f6582, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:57.901829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476efc6-239a-487a-a0b8-2fe3c73caf6c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:57.912455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1122/2231] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:46:57 2022] PUT /identity/v3/projects/5028a97c345b4a76baa7e8e67e3eab2d/users/441c4e2362af45a388d8f3e58e3f6582/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:46:57.922789 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1aa4c-0b98-4dd5-a9fc-49427e57d114 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:57.923073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1aa4c-0b98-4dd5-a9fc-49427e57d114 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:57.923346 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1aa4c-0b98-4dd5-a9fc-49427e57d114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:57.957987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-bbe1aa4c-0b98-4dd5-a9fc-49427e57d114 None None] MFA Rules not processed for user `441c4e2362af45a388d8f3e58e3f6582`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:46:57.959329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-bbe1aa4c-0b98-4dd5-a9fc-49427e57d114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:58.036941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1110/2232] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:46:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:46:58.044912 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476efc6-239a-487a-a0b8-2fe3c73caf6c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:58.046785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6476efc6-239a-487a-a0b8-2fe3c73caf6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:46:58.109642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476efc6-239a-487a-a0b8-2fe3c73caf6c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:58.113400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35aedf-0951-474c-9286-c42706a62ac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:58.114013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35aedf-0951-474c-9286-c42706a62ac9 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:58.114293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35aedf-0951-474c-9286-c42706a62ac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:58.114581 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35aedf-0951-474c-9286-c42706a62ac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:58.124034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35aedf-0951-474c-9286-c42706a62ac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:58.125503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35aedf-0951-474c-9286-c42706a62ac9 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:58.135515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1123/2233] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:46:58.221775 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1aa4c-0b98-4dd5-a9fc-49427e57d114 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:46:58.233452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1aa4c-0b98-4dd5-a9fc-49427e57d114 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:46:58.243695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae7671-7bec-42cc-8afd-1e248c0f48e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:46:58.244677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae7671-7bec-42cc-8afd-1e248c0f48e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:46:58.245053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae7671-7bec-42cc-8afd-1e248c0f48e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:46:58.245575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae7671-7bec-42cc-8afd-1e248c0f48e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:46:58.346689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae7671-7bec-42cc-8afd-1e248c0f48e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:46:58.348777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae7671-7bec-42cc-8afd-1e248c0f48e7 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:46:58.358467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1111/2234] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:46:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:08.131006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=66,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:08.353949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=76 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:08.830626 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35aedf-0951-474c-9286-c42706a62ac9 service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:08.893109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35aedf-0951-474c-9286-c42706a62ac9 service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:08.898245 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:08.898871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:08.899151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:08.899426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] PATH_INFO: `/v3/users/86d208c795024a729f1f95dd07c2e214` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:08.900545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=86d208c795024a729f1f95dd07c2e214)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:08.902503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:08.948502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86d208c795024a729f1f95dd07c2e214'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:08.961085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86d208c795024a729f1f95dd07c2e214'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:08.961742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86d208c795024a729f1f95dd07c2e214'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:08.968460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86d208c795024a729f1f95dd07c2e214'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:08.969120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86d208c795024a729f1f95dd07c2e214'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:08.981818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1124/2235] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:08 2022] DELETE /identity/v3/users/86d208c795024a729f1f95dd07c2e214 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:08.986261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c7b4c-b6fa-4c74-850e-9ae2521d406a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:08.986546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c7b4c-b6fa-4c74-850e-9ae2521d406a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:08.986823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c7b4c-b6fa-4c74-850e-9ae2521d406a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:09.005666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-229c7b4c-b6fa-4c74-850e-9ae2521d406a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:09.006881 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-229c7b4c-b6fa-4c74-850e-9ae2521d406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:09.064103 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1112/2236] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:09.074312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:09.075710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:09.132706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca75c0-d8a1-4382-8041-85b404350cae admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:09.136416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce3fe-7d20-4a85-b73e-b0a2cdcd8d3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:09.137075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce3fe-7d20-4a85-b73e-b0a2cdcd8d3a service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:09.137356 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce3fe-7d20-4a85-b73e-b0a2cdcd8d3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:09.137627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce3fe-7d20-4a85-b73e-b0a2cdcd8d3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:09.146377 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ce3fe-7d20-4a85-b73e-b0a2cdcd8d3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:09.147825 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ce3fe-7d20-4a85-b73e-b0a2cdcd8d3a service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:09.157006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1125/2237] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:09.416726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c7b4c-b6fa-4c74-850e-9ae2521d406a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:09.418245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-229c7b4c-b6fa-4c74-850e-9ae2521d406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:09.430092 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c7b4c-b6fa-4c74-850e-9ae2521d406a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:09.434974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:09.435610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:09.435889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:09.436159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] PATH_INFO: `/v3/projects/304cf96f8c6a4993a1bf3122efb014a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:09.437337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=304cf96f8c6a4993a1bf3122efb014a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:09.439280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:09.485493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304cf96f8c6a4993a1bf3122efb014a9', 'request_id': 'req-57b21971-e62d-4716-86c2-25e22cea9cdd'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:09.490909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304cf96f8c6a4993a1bf3122efb014a9', 'request_id': 'req-57b21971-e62d-4716-86c2-25e22cea9cdd'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:09.498117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304cf96f8c6a4993a1bf3122efb014a9', 'request_id': 'req-57b21971-e62d-4716-86c2-25e22cea9cdd'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:09.509421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] The token cache is being invalidate because project 304cf96f8c6a4993a1bf3122efb014a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:09.509715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:09.510356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:09.511837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1113/2238] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:09 2022] DELETE /identity/v3/projects/304cf96f8c6a4993a1bf3122efb014a9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:09.515826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce3fe-7d20-4a85-b73e-b0a2cdcd8d3a service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:09.525803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce3fe-7d20-4a85-b73e-b0a2cdcd8d3a service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:09.543855 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:09.582910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:09.583530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:09.583811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:09.584090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] PATH_INFO: `/v3/users/441c4e2362af45a388d8f3e58e3f6582` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:09.585247 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=441c4e2362af45a388d8f3e58e3f6582)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:09.587161 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:09.631762 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '441c4e2362af45a388d8f3e58e3f6582'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:09.644029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '441c4e2362af45a388d8f3e58e3f6582'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:09.644684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '441c4e2362af45a388d8f3e58e3f6582'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:09.653081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '441c4e2362af45a388d8f3e58e3f6582'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:09.653765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-1cff736f-a710-448f-80b5-afb7aa281ef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '441c4e2362af45a388d8f3e58e3f6582'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:09.666625 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1126/2239] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:09 2022] DELETE /identity/v3/users/441c4e2362af45a388d8f3e58e3f6582 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:10.030616 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.032174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:10.046314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b21971-e62d-4716-86c2-25e22cea9cdd admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:10.107122 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:10.107779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.108057 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.108325 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] PATH_INFO: `/v3/projects/5028a97c345b4a76baa7e8e67e3eab2d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.109467 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5028a97c345b4a76baa7e8e67e3eab2d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:10.111383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:10.143909 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:10.161504 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5028a97c345b4a76baa7e8e67e3eab2d', 'request_id': 'req-b177d235-0325-45a0-bff4-3438c80ff4fb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:10.167053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5028a97c345b4a76baa7e8e67e3eab2d', 'request_id': 'req-b177d235-0325-45a0-bff4-3438c80ff4fb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:10.175197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5028a97c345b4a76baa7e8e67e3eab2d', 'request_id': 'req-b177d235-0325-45a0-bff4-3438c80ff4fb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:10.186845 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] The token cache is being invalidate because project 5028a97c345b4a76baa7e8e67e3eab2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:10.187130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:10.187779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b177d235-0325-45a0-bff4-3438c80ff4fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:10.189324 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1114/2240] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:10 2022] DELETE /identity/v3/projects/5028a97c345b4a76baa7e8e67e3eab2d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:10.194764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ace2b-f394-4b59-8878-7b98d85f0af8 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.195192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ace2b-f394-4b59-8878-7b98d85f0af8 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.195330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ace2b-f394-4b59-8878-7b98d85f0af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.214557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6b6ace2b-f394-4b59-8878-7b98d85f0af8 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:10.215750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6b6ace2b-f394-4b59-8878-7b98d85f0af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:10.273101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1127/2241] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:10.281228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c05025-4434-4e2d-b31b-25300d2e80d7 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.281453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c05025-4434-4e2d-b31b-25300d2e80d7 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.281734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c05025-4434-4e2d-b31b-25300d2e80d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.301721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a6c05025-4434-4e2d-b31b-25300d2e80d7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:10.302928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a6c05025-4434-4e2d-b31b-25300d2e80d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:10.310958 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1115/2242] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:10.315686 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ace2b-f394-4b59-8878-7b98d85f0af8 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.325301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ace2b-f394-4b59-8878-7b98d85f0af8 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:10.330179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d56a3-214e-4e95-81d6-8c91b16bb173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:10.330794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d56a3-214e-4e95-81d6-8c91b16bb173 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.331074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d56a3-214e-4e95-81d6-8c91b16bb173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.331346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d56a3-214e-4e95-81d6-8c91b16bb173 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.331991 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d56a3-214e-4e95-81d6-8c91b16bb173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:47:10.332331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d56a3-214e-4e95-81d6-8c91b16bb173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:10.333954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d56a3-214e-4e95-81d6-8c91b16bb173 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:10.345263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1128/2243] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:47:10.349833 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c05025-4434-4e2d-b31b-25300d2e80d7 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.359136 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c05025-4434-4e2d-b31b-25300d2e80d7 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:10.364562 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7adc716-591e-4aa0-89e7-bbf13bbf9fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:10.365202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adc716-591e-4aa0-89e7-bbf13bbf9fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.365483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adc716-591e-4aa0-89e7-bbf13bbf9fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.365752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adc716-591e-4aa0-89e7-bbf13bbf9fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.366377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7adc716-591e-4aa0-89e7-bbf13bbf9fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:10.368276 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7adc716-591e-4aa0-89e7-bbf13bbf9fe7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:10.396032 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1116/2244] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:10 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:10.400134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d56a3-214e-4e95-81d6-8c91b16bb173 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.410281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d56a3-214e-4e95-81d6-8c91b16bb173 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:10.435727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:10.467405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4e697-6b7e-45ba-9e03-e15b09347263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:10.468009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4e697-6b7e-45ba-9e03-e15b09347263 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.468291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4e697-6b7e-45ba-9e03-e15b09347263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.468564 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4e697-6b7e-45ba-9e03-e15b09347263 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.469260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4e697-6b7e-45ba-9e03-e15b09347263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:10.471172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4e697-6b7e-45ba-9e03-e15b09347263 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:10.492401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1129/2245] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:10 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:10.496539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7adc716-591e-4aa0-89e7-bbf13bbf9fe7 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.506689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7adc716-591e-4aa0-89e7-bbf13bbf9fe7 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:10.511236 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecc28b-010c-4505-9d3b-e5257e63c5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:10.511829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecc28b-010c-4505-9d3b-e5257e63c5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.512105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecc28b-010c-4505-9d3b-e5257e63c5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.512395 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecc28b-010c-4505-9d3b-e5257e63c5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.513048 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffecc28b-010c-4505-9d3b-e5257e63c5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:10.514636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffecc28b-010c-4505-9d3b-e5257e63c5c3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:10.524312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1117/2246] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:10.528400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4e697-6b7e-45ba-9e03-e15b09347263 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.538757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4e697-6b7e-45ba-9e03-e15b09347263 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:10.543376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2df040-45f0-477f-8b07-307b065f2372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:10.543986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2df040-45f0-477f-8b07-307b065f2372 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.544263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2df040-45f0-477f-8b07-307b065f2372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.544531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2df040-45f0-477f-8b07-307b065f2372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.545196 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2df040-45f0-477f-8b07-307b065f2372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:10.546825 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2df040-45f0-477f-8b07-307b065f2372 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:10.558303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1130/2247] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:10.562530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecc28b-010c-4505-9d3b-e5257e63c5c3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.572262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecc28b-010c-4505-9d3b-e5257e63c5c3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:10.576817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8e5e5-ad6b-444c-8532-7e7ee7678575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:10.577492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8e5e5-ad6b-444c-8532-7e7ee7678575 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.577764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8e5e5-ad6b-444c-8532-7e7ee7678575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.578320 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8e5e5-ad6b-444c-8532-7e7ee7678575 admin admin] PATH_INFO: `/v3/projects/821f412b9c624b56bc97b9a64d99774f/users/c6157479e81144b5ae32a9c0787e419d/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.592418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8e5e5-ad6b-444c-8532-7e7ee7678575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=821f412b9c624b56bc97b9a64d99774f, user_id=c6157479e81144b5ae32a9c0787e419d, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:10.595092 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8e5e5-ad6b-444c-8532-7e7ee7678575 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:10.608030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1118/2248] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:10 2022] PUT /identity/v3/projects/821f412b9c624b56bc97b9a64d99774f/users/c6157479e81144b5ae32a9c0787e419d/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:10.619911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabd995-f8f4-46ec-b00d-f86523752544 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.620201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabd995-f8f4-46ec-b00d-f86523752544 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.620471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabd995-f8f4-46ec-b00d-f86523752544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.648511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-ccabd995-f8f4-46ec-b00d-f86523752544 None None] MFA Rules not processed for user `c6157479e81144b5ae32a9c0787e419d`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:10.649766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ccabd995-f8f4-46ec-b00d-f86523752544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:10.715804 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1131/2249] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:10.720371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8e5e5-ad6b-444c-8532-7e7ee7678575 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.787510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8e5e5-ad6b-444c-8532-7e7ee7678575 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:10.792329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9be33-af03-4776-935c-e1326c94d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:10.792905 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9be33-af03-4776-935c-e1326c94d532 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.793185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9be33-af03-4776-935c-e1326c94d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.793452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9be33-af03-4776-935c-e1326c94d532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.794090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9be33-af03-4776-935c-e1326c94d532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:10.795988 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9be33-af03-4776-935c-e1326c94d532 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:10.822211 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e4c45-4d6f-4e08-84a8-caad97487f6f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.824237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e4c45-4d6f-4e08-84a8-caad97487f6f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.824681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e4c45-4d6f-4e08-84a8-caad97487f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.835294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1119/2250] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:10 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:10.836364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9be33-af03-4776-935c-e1326c94d532 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.850028 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e94e4c45-4d6f-4e08-84a8-caad97487f6f None None] MFA Rules not processed for user `74136098746a4c10b1a74051b75bbd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:10.851397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e94e4c45-4d6f-4e08-84a8-caad97487f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:10.880783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:10.911354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9be33-af03-4776-935c-e1326c94d532 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:10.917791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4437ee-2d5e-49f2-9d04-14e7d267acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:10.918778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4437ee-2d5e-49f2-9d04-14e7d267acfe admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.919223 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4437ee-2d5e-49f2-9d04-14e7d267acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.919836 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4437ee-2d5e-49f2-9d04-14e7d267acfe admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.920689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4437ee-2d5e-49f2-9d04-14e7d267acfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:10.922891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4437ee-2d5e-49f2-9d04-14e7d267acfe admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:10.934159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1132/2251] 10.222.0.17 () {58 vars in 967 bytes} [Tue Oct 18 11:47:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:47:10.947645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1120/2252] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:10 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:10.949826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e4c45-4d6f-4e08-84a8-caad97487f6f None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.952044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e94e4c45-4d6f-4e08-84a8-caad97487f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:10.952466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4437ee-2d5e-49f2-9d04-14e7d267acfe admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:10.965376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4437ee-2d5e-49f2-9d04-14e7d267acfe admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:10.970634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bde40-d942-45a0-8e53-8c12320704f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:10.972096 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bde40-d942-45a0-8e53-8c12320704f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:10.972525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bde40-d942-45a0-8e53-8c12320704f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:10.972986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bde40-d942-45a0-8e53-8c12320704f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:10.973997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bde40-d942-45a0-8e53-8c12320704f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:10.976575 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bde40-d942-45a0-8e53-8c12320704f4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:10.989697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1121/2253] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:10.998308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bde40-d942-45a0-8e53-8c12320704f4 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:11.002426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:11.012007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bde40-d942-45a0-8e53-8c12320704f4 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:11.019520 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ca5a9-d450-4241-ae7b-4281b2b954af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:11.022167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ca5a9-d450-4241-ae7b-4281b2b954af admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:11.022167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ca5a9-d450-4241-ae7b-4281b2b954af admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:11.022167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ca5a9-d450-4241-ae7b-4281b2b954af admin admin] PATH_INFO: `/v3/projects/4b2df078fa154e3386063c81ec735c7a/users/16cd71c1f8104dc0a386d8c705ffe97c/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:11.036091 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e4c45-4d6f-4e08-84a8-caad97487f6f None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:11.041794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb62c9-cadf-47d7-a11b-d832b0665c2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4be9de8292214872a59c1466f7513539', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:11.042178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ca5a9-d450-4241-ae7b-4281b2b954af admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2df078fa154e3386063c81ec735c7a, user_id=16cd71c1f8104dc0a386d8c705ffe97c, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:11.042429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb62c9-cadf-47d7-a11b-d832b0665c2c service swift] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:11.042651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb62c9-cadf-47d7-a11b-d832b0665c2c service swift] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:11.042927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb62c9-cadf-47d7-a11b-d832b0665c2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:11.045814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ca5a9-d450-4241-ae7b-4281b2b954af admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:11.052676 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb62c9-cadf-47d7-a11b-d832b0665c2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:11.054100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb62c9-cadf-47d7-a11b-d832b0665c2c service swift] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:11.065590 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1133/2254] 10.222.0.17 () {60 vars in 1384 bytes} [Tue Oct 18 11:47:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 18 11:47:11.069250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1122/2255] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:10 2022] PUT /identity/v3/projects/4b2df078fa154e3386063c81ec735c7a/users/16cd71c1f8104dc0a386d8c705ffe97c/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:11.070711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb62c9-cadf-47d7-a11b-d832b0665c2c service swift] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:11.083691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb62c9-cadf-47d7-a11b-d832b0665c2c service swift] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:11.171489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c5128-5d23-4ac8-acc2-5bb2b1756517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:11.172488 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c5128-5d23-4ac8-acc2-5bb2b1756517 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:11.173012 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c5128-5d23-4ac8-acc2-5bb2b1756517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:11.173453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c5128-5d23-4ac8-acc2-5bb2b1756517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:11.174375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c5128-5d23-4ac8-acc2-5bb2b1756517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:11.176494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c5128-5d23-4ac8-acc2-5bb2b1756517 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:11.193070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1134/2256] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:11.193962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ca5a9-d450-4241-ae7b-4281b2b954af admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:11.206223 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ca5a9-d450-4241-ae7b-4281b2b954af admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:11.220211 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c22e60-92b6-45de-b6d4-6b6ad6052052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:11.223107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c22e60-92b6-45de-b6d4-6b6ad6052052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:11.228420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c22e60-92b6-45de-b6d4-6b6ad6052052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:11.228899 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c22e60-92b6-45de-b6d4-6b6ad6052052 admin admin] PATH_INFO: `/v3/projects/4b2df078fa154e3386063c81ec735c7a/users/16cd71c1f8104dc0a386d8c705ffe97c/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:11.232867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c22e60-92b6-45de-b6d4-6b6ad6052052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2df078fa154e3386063c81ec735c7a, user_id=16cd71c1f8104dc0a386d8c705ffe97c, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:11.236743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c22e60-92b6-45de-b6d4-6b6ad6052052 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:11.249897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1123/2257] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:11 2022] PUT /identity/v3/projects/4b2df078fa154e3386063c81ec735c7a/users/16cd71c1f8104dc0a386d8c705ffe97c/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:11.260730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08a19-ae26-40fd-8158-cef44d48b25b None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:11.261217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08a19-ae26-40fd-8158-cef44d48b25b None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:11.261519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08a19-ae26-40fd-8158-cef44d48b25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:11.295199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-32b08a19-ae26-40fd-8158-cef44d48b25b None None] MFA Rules not processed for user `16cd71c1f8104dc0a386d8c705ffe97c`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:11.296907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-32b08a19-ae26-40fd-8158-cef44d48b25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:11.380536 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1135/2258] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:11.397674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c22e60-92b6-45de-b6d4-6b6ad6052052 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:11.399578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-77c22e60-92b6-45de-b6d4-6b6ad6052052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:11.475995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c22e60-92b6-45de-b6d4-6b6ad6052052 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:11.483898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b399b-32d4-400a-a427-81cdb67051aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:11.483898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b399b-32d4-400a-a427-81cdb67051aa service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:11.484406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b399b-32d4-400a-a427-81cdb67051aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:11.484406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b399b-32d4-400a-a427-81cdb67051aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:11.495777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b399b-32d4-400a-a427-81cdb67051aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:11.497361 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b399b-32d4-400a-a427-81cdb67051aa service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:11.508718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1124/2259] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:12.205567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b08a19-ae26-40fd-8158-cef44d48b25b None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:12.277022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:12.292642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b08a19-ae26-40fd-8158-cef44d48b25b None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:12.300909 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:12.301693 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:12.302029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:12.302237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] PATH_INFO: `/v3/users/c6157479e81144b5ae32a9c0787e419d` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:12.304144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6157479e81144b5ae32a9c0787e419d)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:12.306552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:12.388654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6157479e81144b5ae32a9c0787e419d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:12.408489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6157479e81144b5ae32a9c0787e419d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:12.410448 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6157479e81144b5ae32a9c0787e419d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:12.419531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6157479e81144b5ae32a9c0787e419d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:12.420249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6157479e81144b5ae32a9c0787e419d'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:12.437924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1136/2260] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:12 2022] DELETE /identity/v3/users/c6157479e81144b5ae32a9c0787e419d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:12.442372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee34e2e-8411-4923-800d-d2628235a194 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:12.442774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee34e2e-8411-4923-800d-d2628235a194 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:12.443296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee34e2e-8411-4923-800d-d2628235a194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:12.468214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-0ee34e2e-8411-4923-800d-d2628235a194 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:12.469528 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0ee34e2e-8411-4923-800d-d2628235a194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:12.537374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1125/2261] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:47:12.547406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:12.548903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:12.609125 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770d6e7-6033-4e7e-b5eb-4fbe22f4f58f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:12.613004 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044ebcd-bb84-4d39-9ee6-e99c39968cf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:12.613768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044ebcd-bb84-4d39-9ee6-e99c39968cf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:12.614094 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044ebcd-bb84-4d39-9ee6-e99c39968cf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:12.614365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044ebcd-bb84-4d39-9ee6-e99c39968cf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:12.624044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044ebcd-bb84-4d39-9ee6-e99c39968cf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:12.625518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044ebcd-bb84-4d39-9ee6-e99c39968cf1 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:12.636084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1137/2262] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:12.906602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee34e2e-8411-4923-800d-d2628235a194 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:12.908597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0ee34e2e-8411-4923-800d-d2628235a194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:12.925708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee34e2e-8411-4923-800d-d2628235a194 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:12.939003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:12.939757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:12.940901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:12.941604 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] PATH_INFO: `/v3/projects/821f412b9c624b56bc97b9a64d99774f` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:12.943470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=821f412b9c624b56bc97b9a64d99774f)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:12.945820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:12.964477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:12.995481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '821f412b9c624b56bc97b9a64d99774f', 'request_id': 'req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.001019 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '821f412b9c624b56bc97b9a64d99774f', 'request_id': 'req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.008610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '821f412b9c624b56bc97b9a64d99774f', 'request_id': 'req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.019636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] The token cache is being invalidate because project 821f412b9c624b56bc97b9a64d99774f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:13.019914 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.020534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.022020 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1126/2263] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:12 2022] DELETE /identity/v3/projects/821f412b9c624b56bc97b9a64d99774f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:13.025972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044ebcd-bb84-4d39-9ee6-e99c39968cf1 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:13.035428 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044ebcd-bb84-4d39-9ee6-e99c39968cf1 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:13.090326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:13.090945 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:13.091221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:13.091491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] PATH_INFO: `/v3/users/16cd71c1f8104dc0a386d8c705ffe97c` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:13.092593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16cd71c1f8104dc0a386d8c705ffe97c)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:13.094533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:13.109566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:13.140176 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16cd71c1f8104dc0a386d8c705ffe97c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.152130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16cd71c1f8104dc0a386d8c705ffe97c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.152740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16cd71c1f8104dc0a386d8c705ffe97c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.159523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16cd71c1f8104dc0a386d8c705ffe97c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.160155 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-7c853be4-8001-4303-b8f4-2b5b42e6a769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16cd71c1f8104dc0a386d8c705ffe97c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.172547 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1138/2264] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:13 2022] DELETE /identity/v3/users/16cd71c1f8104dc0a386d8c705ffe97c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:13.451549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:13.453301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:13.466726 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed095c-869c-43af-b8b5-b4a9b30fd7c1 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:13.531559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:13.532194 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:13.532485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:13.532739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] PATH_INFO: `/v3/projects/4b2df078fa154e3386063c81ec735c7a` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:13.533959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b2df078fa154e3386063c81ec735c7a)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:13.535857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:13.593329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b2df078fa154e3386063c81ec735c7a', 'request_id': 'req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.602826 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b2df078fa154e3386063c81ec735c7a', 'request_id': 'req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.614804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b2df078fa154e3386063c81ec735c7a', 'request_id': 'req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.626750 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] The token cache is being invalidate because project 4b2df078fa154e3386063c81ec735c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:13.627039 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.628202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2d8ef09-8f60-4dec-a72b-1aa5535189f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:13.629727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1127/2265] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:13 2022] DELETE /identity/v3/projects/4b2df078fa154e3386063c81ec735c7a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:13.635304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c501c-56e5-4be3-a7ad-c12513169be1 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:13.635798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c501c-56e5-4be3-a7ad-c12513169be1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:13.635900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c501c-56e5-4be3-a7ad-c12513169be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:13.656508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-a87c501c-56e5-4be3-a7ad-c12513169be1 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:13.658751 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a87c501c-56e5-4be3-a7ad-c12513169be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:13.717819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1139/2266] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:13.725835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3be9a-8082-468c-ad05-a8ff84d84d11 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:13.726120 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3be9a-8082-468c-ad05-a8ff84d84d11 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:13.726393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3be9a-8082-468c-ad05-a8ff84d84d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:13.747854 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-e2d3be9a-8082-468c-ad05-a8ff84d84d11 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:13.749154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e2d3be9a-8082-468c-ad05-a8ff84d84d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:13.757998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1128/2267] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:13.762533 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c501c-56e5-4be3-a7ad-c12513169be1 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:13.772165 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c501c-56e5-4be3-a7ad-c12513169be1 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:13.776782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794f278-731d-4a66-bd43-5e13f74fb02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:13.777419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794f278-731d-4a66-bd43-5e13f74fb02d admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:13.777700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794f278-731d-4a66-bd43-5e13f74fb02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:13.777970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794f278-731d-4a66-bd43-5e13f74fb02d admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:13.778646 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794f278-731d-4a66-bd43-5e13f74fb02d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:47:13.778948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794f278-731d-4a66-bd43-5e13f74fb02d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:13.780531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794f278-731d-4a66-bd43-5e13f74fb02d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:13.792776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1140/2268] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:47:13.797078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3be9a-8082-468c-ad05-a8ff84d84d11 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:13.806572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3be9a-8082-468c-ad05-a8ff84d84d11 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:13.811254 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283f450-a021-48d5-adf7-405efef257d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:13.811857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283f450-a021-48d5-adf7-405efef257d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:13.812133 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283f450-a021-48d5-adf7-405efef257d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:13.812403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283f450-a021-48d5-adf7-405efef257d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:13.813060 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283f450-a021-48d5-adf7-405efef257d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:13.815010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283f450-a021-48d5-adf7-405efef257d3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:13.846009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1129/2269] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:13 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:13.847706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794f278-731d-4a66-bd43-5e13f74fb02d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:13.858076 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794f278-731d-4a66-bd43-5e13f74fb02d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:13.917543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a15b9-c538-46b9-8371-d02d465ee65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:13.918162 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a15b9-c538-46b9-8371-d02d465ee65d admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:13.918442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a15b9-c538-46b9-8371-d02d465ee65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:13.918719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a15b9-c538-46b9-8371-d02d465ee65d admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:13.919379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a15b9-c538-46b9-8371-d02d465ee65d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:13.921310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a15b9-c538-46b9-8371-d02d465ee65d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:13.944874 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1141/2270] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:13 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:47:13.948006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283f450-a021-48d5-adf7-405efef257d3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:13.958357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:13.962969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283f450-a021-48d5-adf7-405efef257d3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:13.969387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b7c77-9298-4ccc-927b-fe13cc229ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:13.970085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b7c77-9298-4ccc-927b-fe13cc229ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:13.970603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b7c77-9298-4ccc-927b-fe13cc229ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:13.971955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b7c77-9298-4ccc-927b-fe13cc229ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:13.972742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b7c77-9298-4ccc-927b-fe13cc229ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:13.974740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b7c77-9298-4ccc-927b-fe13cc229ba1 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:13.988193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1130/2271] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:13.989766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a15b9-c538-46b9-8371-d02d465ee65d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:14.000246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a15b9-c538-46b9-8371-d02d465ee65d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:14.004974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795dac5-51c9-4d28-b00b-45bf97da15ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:14.005578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795dac5-51c9-4d28-b00b-45bf97da15ce admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.005856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795dac5-51c9-4d28-b00b-45bf97da15ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.006128 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795dac5-51c9-4d28-b00b-45bf97da15ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.006776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795dac5-51c9-4d28-b00b-45bf97da15ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:14.008357 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795dac5-51c9-4d28-b00b-45bf97da15ce admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:14.013693 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:14.020986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1142/2272] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:14.024430 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b7c77-9298-4ccc-927b-fe13cc229ba1 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:14.035551 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b7c77-9298-4ccc-927b-fe13cc229ba1 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:14.040143 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84c445-8e88-4b7c-9107-c2313ff1b69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:14.040801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c445-8e88-4b7c-9107-c2313ff1b69a admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.041111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c445-8e88-4b7c-9107-c2313ff1b69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.041384 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c445-8e88-4b7c-9107-c2313ff1b69a admin admin] PATH_INFO: `/v3/projects/2f3951ed199c46d49286ca2cfe835a2d/users/6c92135d24b54491b0baa474c9ffb0eb/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.057743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84c445-8e88-4b7c-9107-c2313ff1b69a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3951ed199c46d49286ca2cfe835a2d, user_id=6c92135d24b54491b0baa474c9ffb0eb, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:14.060385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84c445-8e88-4b7c-9107-c2313ff1b69a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:14.073188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1131/2273] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:14 2022] PUT /identity/v3/projects/2f3951ed199c46d49286ca2cfe835a2d/users/6c92135d24b54491b0baa474c9ffb0eb/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:14.085182 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96250971-aef8-442e-89ee-4edcfd80a1d6 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.085465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96250971-aef8-442e-89ee-4edcfd80a1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.085737 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96250971-aef8-442e-89ee-4edcfd80a1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.115903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-96250971-aef8-442e-89ee-4edcfd80a1d6 None None] MFA Rules not processed for user `6c92135d24b54491b0baa474c9ffb0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:14.117207 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-96250971-aef8-442e-89ee-4edcfd80a1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:14.187984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1143/2274] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:14.193657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84c445-8e88-4b7c-9107-c2313ff1b69a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:14.254981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84c445-8e88-4b7c-9107-c2313ff1b69a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:14.259843 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f944eab-1d6a-42cf-ab39-6cefc2b301e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:14.260468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f944eab-1d6a-42cf-ab39-6cefc2b301e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.260744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f944eab-1d6a-42cf-ab39-6cefc2b301e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.261042 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f944eab-1d6a-42cf-ab39-6cefc2b301e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.261780 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f944eab-1d6a-42cf-ab39-6cefc2b301e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:14.263758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f944eab-1d6a-42cf-ab39-6cefc2b301e2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:14.289795 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1132/2275] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:14 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:47:14.294210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96250971-aef8-442e-89ee-4edcfd80a1d6 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:14.305579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96250971-aef8-442e-89ee-4edcfd80a1d6 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:14.367959 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba2c98-4f87-45b2-89c8-ea48df154d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:14.368571 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba2c98-4f87-45b2-89c8-ea48df154d7d admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.368940 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba2c98-4f87-45b2-89c8-ea48df154d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.369148 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba2c98-4f87-45b2-89c8-ea48df154d7d admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.369784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba2c98-4f87-45b2-89c8-ea48df154d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:14.371688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba2c98-4f87-45b2-89c8-ea48df154d7d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:14.392854 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1144/2276] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:14 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:14.396893 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f944eab-1d6a-42cf-ab39-6cefc2b301e2 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:14.407146 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f944eab-1d6a-42cf-ab39-6cefc2b301e2 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:14.412443 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc902a-5657-4837-a481-ccc261338ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:14.413093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc902a-5657-4837-a481-ccc261338ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.413373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc902a-5657-4837-a481-ccc261338ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.413641 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc902a-5657-4837-a481-ccc261338ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.414311 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc902a-5657-4837-a481-ccc261338ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:14.415997 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc902a-5657-4837-a481-ccc261338ff0 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:14.427115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1133/2277] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:14.431118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba2c98-4f87-45b2-89c8-ea48df154d7d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:14.441095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba2c98-4f87-45b2-89c8-ea48df154d7d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:14.445768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0aff06-959c-451c-aeac-010e0f7e80ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:14.446407 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0aff06-959c-451c-aeac-010e0f7e80ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.446688 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0aff06-959c-451c-aeac-010e0f7e80ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.446962 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0aff06-959c-451c-aeac-010e0f7e80ef admin admin] PATH_INFO: `/v3/projects/fb8c2174597347608692e2b398ac4eb6/users/05e2c2123c8040c1a2b979c55329e65c/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.463304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0aff06-959c-451c-aeac-010e0f7e80ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8c2174597347608692e2b398ac4eb6, user_id=05e2c2123c8040c1a2b979c55329e65c, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:14.466076 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0aff06-959c-451c-aeac-010e0f7e80ef admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:14.479030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1145/2278] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:14 2022] PUT /identity/v3/projects/fb8c2174597347608692e2b398ac4eb6/users/05e2c2123c8040c1a2b979c55329e65c/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:14.483105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc902a-5657-4837-a481-ccc261338ff0 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:14.493200 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc902a-5657-4837-a481-ccc261338ff0 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:14.547715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd74b7d-3709-46c5-a2d0-5952eb74b794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:14.548309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd74b7d-3709-46c5-a2d0-5952eb74b794 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.548582 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd74b7d-3709-46c5-a2d0-5952eb74b794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.548868 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd74b7d-3709-46c5-a2d0-5952eb74b794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.549502 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd74b7d-3709-46c5-a2d0-5952eb74b794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:14.551105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd74b7d-3709-46c5-a2d0-5952eb74b794 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:14.560453 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1134/2279] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:14.564632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0aff06-959c-451c-aeac-010e0f7e80ef admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:14.574822 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0aff06-959c-451c-aeac-010e0f7e80ef admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:14.579752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e065a3-934d-4cb9-8e91-76bc5baa3917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:14.580393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e065a3-934d-4cb9-8e91-76bc5baa3917 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.581228 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e065a3-934d-4cb9-8e91-76bc5baa3917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.581504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e065a3-934d-4cb9-8e91-76bc5baa3917 admin admin] PATH_INFO: `/v3/projects/fb8c2174597347608692e2b398ac4eb6/users/05e2c2123c8040c1a2b979c55329e65c/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.583426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e065a3-934d-4cb9-8e91-76bc5baa3917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8c2174597347608692e2b398ac4eb6, user_id=05e2c2123c8040c1a2b979c55329e65c, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:14.586109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e065a3-934d-4cb9-8e91-76bc5baa3917 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:14.596661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1146/2280] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:14 2022] PUT /identity/v3/projects/fb8c2174597347608692e2b398ac4eb6/users/05e2c2123c8040c1a2b979c55329e65c/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:14.606896 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd97f4f-5783-41fd-86bf-59bdbda611de None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.607367 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd97f4f-5783-41fd-86bf-59bdbda611de None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.607496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd97f4f-5783-41fd-86bf-59bdbda611de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.634620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-3bd97f4f-5783-41fd-86bf-59bdbda611de None None] MFA Rules not processed for user `05e2c2123c8040c1a2b979c55329e65c`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:14.635820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-3bd97f4f-5783-41fd-86bf-59bdbda611de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:14.665571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:14.704944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1135/2281] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:14.716051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e065a3-934d-4cb9-8e91-76bc5baa3917 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:14.717901 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-09e065a3-934d-4cb9-8e91-76bc5baa3917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:14.764788 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:14.777294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e065a3-934d-4cb9-8e91-76bc5baa3917 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:14.780914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfaa42-74ed-48ed-b057-5bd83b51a4fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:14.781507 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfaa42-74ed-48ed-b057-5bd83b51a4fb service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:14.781784 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfaa42-74ed-48ed-b057-5bd83b51a4fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:14.782050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfaa42-74ed-48ed-b057-5bd83b51a4fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:14.838782 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfaa42-74ed-48ed-b057-5bd83b51a4fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:14.840202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfaa42-74ed-48ed-b057-5bd83b51a4fb service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:14.849453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1147/2282] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:17.380690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd97f4f-5783-41fd-86bf-59bdbda611de None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:17.441044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd97f4f-5783-41fd-86bf-59bdbda611de None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:17.445491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:17.446108 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:17.446380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:17.446645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] PATH_INFO: `/v3/users/6c92135d24b54491b0baa474c9ffb0eb` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:17.447733 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c92135d24b54491b0baa474c9ffb0eb)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:17.449670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:17.495284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c92135d24b54491b0baa474c9ffb0eb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:17.502872 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c92135d24b54491b0baa474c9ffb0eb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:17.503535 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c92135d24b54491b0baa474c9ffb0eb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:17.515340 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c92135d24b54491b0baa474c9ffb0eb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:17.522387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c92135d24b54491b0baa474c9ffb0eb'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:17.528155 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1136/2283] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:17 2022] DELETE /identity/v3/users/6c92135d24b54491b0baa474c9ffb0eb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:17.532590 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f5ac-44d8-475f-9b2b-6eab35c17319 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:17.533080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f5ac-44d8-475f-9b2b-6eab35c17319 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:17.533205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f5ac-44d8-475f-9b2b-6eab35c17319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:17.552288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-4024f5ac-44d8-475f-9b2b-6eab35c17319 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:17.553513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-4024f5ac-44d8-475f-9b2b-6eab35c17319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:17.610601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1148/2284] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:17.620863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:17.622292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:17.669063 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:17.681923 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bb467-d8b8-4476-bf6a-f1b061ee9f67 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:17.685598 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679ca81-a1ad-45a3-9d13-24ffabbabc11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:17.686222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679ca81-a1ad-45a3-9d13-24ffabbabc11 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:17.686483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679ca81-a1ad-45a3-9d13-24ffabbabc11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:17.686759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679ca81-a1ad-45a3-9d13-24ffabbabc11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:17.696336 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679ca81-a1ad-45a3-9d13-24ffabbabc11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:17.697736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679ca81-a1ad-45a3-9d13-24ffabbabc11 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:17.707424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1137/2285] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:17.942135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024f5ac-44d8-475f-9b2b-6eab35c17319 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:17.952122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024f5ac-44d8-475f-9b2b-6eab35c17319 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:17.956872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5e603-5be6-4d17-91ad-9453bde24463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:17.957487 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5e603-5be6-4d17-91ad-9453bde24463 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:17.957767 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5e603-5be6-4d17-91ad-9453bde24463 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:17.958132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5e603-5be6-4d17-91ad-9453bde24463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:17.959766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-28f5e603-5be6-4d17-91ad-9453bde24463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:17.973462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5e603-5be6-4d17-91ad-9453bde24463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:17.974987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5e603-5be6-4d17-91ad-9453bde24463 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:17.982470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679ca81-a1ad-45a3-9d13-24ffabbabc11 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:17.983992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d679ca81-a1ad-45a3-9d13-24ffabbabc11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:17.987933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1149/2286] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:18.004417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679ca81-a1ad-45a3-9d13-24ffabbabc11 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:18.010568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:18.011851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:18.012046 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:18.012361 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] PATH_INFO: `/v3/projects/2f3951ed199c46d49286ca2cfe835a2d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:18.013506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f3951ed199c46d49286ca2cfe835a2d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:18.015460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:18.067069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f3951ed199c46d49286ca2cfe835a2d', 'request_id': 'req-c55e0768-c385-4262-a18b-8498111b40b5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.073222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f3951ed199c46d49286ca2cfe835a2d', 'request_id': 'req-c55e0768-c385-4262-a18b-8498111b40b5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.081062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f3951ed199c46d49286ca2cfe835a2d', 'request_id': 'req-c55e0768-c385-4262-a18b-8498111b40b5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.092633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] The token cache is being invalidate because project 2f3951ed199c46d49286ca2cfe835a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:18.092943 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.093653 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.095358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1138/2287] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:17 2022] DELETE /identity/v3/projects/2f3951ed199c46d49286ca2cfe835a2d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:47:18.099734 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5e603-5be6-4d17-91ad-9453bde24463 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:18.109466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5e603-5be6-4d17-91ad-9453bde24463 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:18.167896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:18.168553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:18.168857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:18.169139 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] PATH_INFO: `/v3/users/05e2c2123c8040c1a2b979c55329e65c` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:18.171145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e2c2123c8040c1a2b979c55329e65c)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:18.173229 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:18.207174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:18.227394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e2c2123c8040c1a2b979c55329e65c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.240552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e2c2123c8040c1a2b979c55329e65c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.241566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e2c2123c8040c1a2b979c55329e65c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.248520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e2c2123c8040c1a2b979c55329e65c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.249193 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-26595674-d54b-4013-b864-45257e170fb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e2c2123c8040c1a2b979c55329e65c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.263124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1150/2288] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:18 2022] DELETE /identity/v3/users/05e2c2123c8040c1a2b979c55329e65c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:18.531239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:18.533425 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:18.549884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e0768-c385-4262-a18b-8498111b40b5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:18.611639 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:18.612482 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:18.612889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:18.613272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] PATH_INFO: `/v3/projects/fb8c2174597347608692e2b398ac4eb6` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:18.614542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb8c2174597347608692e2b398ac4eb6)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:18.616592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:18.664781 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb8c2174597347608692e2b398ac4eb6', 'request_id': 'req-12233afd-610f-4096-bc7a-0e8d44b90a3f'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.670483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb8c2174597347608692e2b398ac4eb6', 'request_id': 'req-12233afd-610f-4096-bc7a-0e8d44b90a3f'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.677777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb8c2174597347608692e2b398ac4eb6', 'request_id': 'req-12233afd-610f-4096-bc7a-0e8d44b90a3f'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.687635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:18.690534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] The token cache is being invalidate because project fb8c2174597347608692e2b398ac4eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:18.690819 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.691960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-12233afd-610f-4096-bc7a-0e8d44b90a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:18.694214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1139/2289] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:18 2022] DELETE /identity/v3/projects/fb8c2174597347608692e2b398ac4eb6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:18.699529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c94ae7-795e-4c28-b993-adbb012f6a64 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:18.699529 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c94ae7-795e-4c28-b993-adbb012f6a64 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:18.700086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c94ae7-795e-4c28-b993-adbb012f6a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:18.719936 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-59c94ae7-795e-4c28-b993-adbb012f6a64 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:18.721281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-59c94ae7-795e-4c28-b993-adbb012f6a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:18.783774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1151/2290] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:18.792499 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94717093-aa06-4cb3-9cb0-18e9e9a804ba None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:18.792797 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94717093-aa06-4cb3-9cb0-18e9e9a804ba None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:18.793102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94717093-aa06-4cb3-9cb0-18e9e9a804ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:18.812414 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-94717093-aa06-4cb3-9cb0-18e9e9a804ba None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:18.813652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-94717093-aa06-4cb3-9cb0-18e9e9a804ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:18.822386 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1140/2291] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:18.826757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c94ae7-795e-4c28-b993-adbb012f6a64 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:18.838430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c94ae7-795e-4c28-b993-adbb012f6a64 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:18.843944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7033f-7a59-4185-91b3-7b7478548486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:18.844559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7033f-7a59-4185-91b3-7b7478548486 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:18.844869 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7033f-7a59-4185-91b3-7b7478548486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:18.845143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7033f-7a59-4185-91b3-7b7478548486 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:18.845814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7033f-7a59-4185-91b3-7b7478548486 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:47:18.846121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7033f-7a59-4185-91b3-7b7478548486 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:18.847715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7033f-7a59-4185-91b3-7b7478548486 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:18.863182 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1152/2292] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:47:18.875319 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94717093-aa06-4cb3-9cb0-18e9e9a804ba None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:18.885686 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94717093-aa06-4cb3-9cb0-18e9e9a804ba None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:18.890553 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f57760-8c20-4353-bca2-40c18242e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:18.891239 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f57760-8c20-4353-bca2-40c18242e071 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:18.891588 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f57760-8c20-4353-bca2-40c18242e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:18.891924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f57760-8c20-4353-bca2-40c18242e071 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:18.892621 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f57760-8c20-4353-bca2-40c18242e071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:18.895210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f57760-8c20-4353-bca2-40c18242e071 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:18.926500 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1141/2293] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:18 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:18.932861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7033f-7a59-4185-91b3-7b7478548486 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:18.951552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7033f-7a59-4185-91b3-7b7478548486 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:19.018612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464d0d1-9caa-4dba-a055-67c84f5e75f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:19.019325 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464d0d1-9caa-4dba-a055-67c84f5e75f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:19.019732 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464d0d1-9caa-4dba-a055-67c84f5e75f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:19.020014 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464d0d1-9caa-4dba-a055-67c84f5e75f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:19.020726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464d0d1-9caa-4dba-a055-67c84f5e75f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:19.022730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464d0d1-9caa-4dba-a055-67c84f5e75f8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:19.050280 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1153/2294] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:18 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:19.054595 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f57760-8c20-4353-bca2-40c18242e071 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:19.066087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f57760-8c20-4353-bca2-40c18242e071 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:19.070689 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c4158-b8b1-47fc-b420-8418f35f6bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:19.071292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c4158-b8b1-47fc-b420-8418f35f6bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:19.071568 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c4158-b8b1-47fc-b420-8418f35f6bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:19.071837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c4158-b8b1-47fc-b420-8418f35f6bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:19.072472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c4158-b8b1-47fc-b420-8418f35f6bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:19.074102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c4158-b8b1-47fc-b420-8418f35f6bc3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:19.085739 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1142/2295] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:47:19.089752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464d0d1-9caa-4dba-a055-67c84f5e75f8 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:19.102799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464d0d1-9caa-4dba-a055-67c84f5e75f8 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:19.109508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee6bb3-65b3-41c9-82a9-14f6ff80a0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:19.110295 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee6bb3-65b3-41c9-82a9-14f6ff80a0ab admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:19.110645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee6bb3-65b3-41c9-82a9-14f6ff80a0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:19.110983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee6bb3-65b3-41c9-82a9-14f6ff80a0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:19.111765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee6bb3-65b3-41c9-82a9-14f6ff80a0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:19.113450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee6bb3-65b3-41c9-82a9-14f6ff80a0ab admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:19.128666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1154/2296] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:19.130840 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c4158-b8b1-47fc-b420-8418f35f6bc3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:19.143493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c4158-b8b1-47fc-b420-8418f35f6bc3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:19.148451 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbdcd9-08e3-48e6-8b70-3c70779290e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:19.149166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbdcd9-08e3-48e6-8b70-3c70779290e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:19.149439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbdcd9-08e3-48e6-8b70-3c70779290e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:19.149705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbdcd9-08e3-48e6-8b70-3c70779290e3 admin admin] PATH_INFO: `/v3/projects/30577d476e034fb39450ad9ee6a0e4ed/users/e4ed95feeb734b609373e251e9796bdc/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:19.166917 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfbdcd9-08e3-48e6-8b70-3c70779290e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30577d476e034fb39450ad9ee6a0e4ed, user_id=e4ed95feeb734b609373e251e9796bdc, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:19.169650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfbdcd9-08e3-48e6-8b70-3c70779290e3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:19.195634 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1143/2297] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:19 2022] PUT /identity/v3/projects/30577d476e034fb39450ad9ee6a0e4ed/users/e4ed95feeb734b609373e251e9796bdc/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:19.206679 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765014f2-0795-471f-9781-84ee62c162ca None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:19.206964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765014f2-0795-471f-9781-84ee62c162ca None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:19.208111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765014f2-0795-471f-9781-84ee62c162ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:19.227021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:19.240416 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-765014f2-0795-471f-9781-84ee62c162ca None None] MFA Rules not processed for user `e4ed95feeb734b609373e251e9796bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:19.243189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-765014f2-0795-471f-9781-84ee62c162ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:19.315513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1155/2298] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:19.329769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbdcd9-08e3-48e6-8b70-3c70779290e3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:19.332400 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2dfbdcd9-08e3-48e6-8b70-3c70779290e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:19.404030 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbdcd9-08e3-48e6-8b70-3c70779290e3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:19.409863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d822588-0121-4d3a-b801-fb6870596d79 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d2a655f4d6f45039d641133e3b56592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:19.410497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d822588-0121-4d3a-b801-fb6870596d79 service glance] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:19.410769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d822588-0121-4d3a-b801-fb6870596d79 service glance] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:19.411034 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d822588-0121-4d3a-b801-fb6870596d79 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:19.420201 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d822588-0121-4d3a-b801-fb6870596d79 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:19.421670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d822588-0121-4d3a-b801-fb6870596d79 service glance] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:19.432254 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1144/2299] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:21.388100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765014f2-0795-471f-9781-84ee62c162ca None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:21.389558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-765014f2-0795-471f-9781-84ee62c162ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:21.461827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765014f2-0795-471f-9781-84ee62c162ca None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:21.466408 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:21.467151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:21.467502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:21.467857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] PATH_INFO: `/v3/users/916699a212eb4d8aa945f299ac0d4c96` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:21.469188 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916699a212eb4d8aa945f299ac0d4c96)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:21.471844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:21.518369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916699a212eb4d8aa945f299ac0d4c96'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:21.530143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916699a212eb4d8aa945f299ac0d4c96'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:21.530816 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916699a212eb4d8aa945f299ac0d4c96'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:21.539131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916699a212eb4d8aa945f299ac0d4c96'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:21.539973 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-69cc3c44-7b3d-41f0-bf24-6a40fafc1b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916699a212eb4d8aa945f299ac0d4c96'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:21.553531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1156/2300] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:21 2022] DELETE /identity/v3/users/916699a212eb4d8aa945f299ac0d4c96 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:21.739395 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d822588-0121-4d3a-b801-fb6870596d79 service glance] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:21.750970 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d822588-0121-4d3a-b801-fb6870596d79 service glance] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:21.809848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:21.810495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:21.810766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:21.811034 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] PATH_INFO: `/v3/projects/633afaabbbff47a4a99ebc997e67b01a` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:21.812167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=633afaabbbff47a4a99ebc997e67b01a)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:21.814104 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:21.856889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:21.866963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633afaabbbff47a4a99ebc997e67b01a', 'request_id': 'req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:21.872880 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633afaabbbff47a4a99ebc997e67b01a', 'request_id': 'req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:21.880195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633afaabbbff47a4a99ebc997e67b01a', 'request_id': 'req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:21.891509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] The token cache is being invalidate because project 633afaabbbff47a4a99ebc997e67b01a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:21.891857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:21.892500 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-c2b6a612-e67f-435a-89cc-2f93d3bae9ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:21.894053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1145/2301] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:21 2022] DELETE /identity/v3/projects/633afaabbbff47a4a99ebc997e67b01a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:21.899121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99922895-5ac1-4238-86e1-829a5e5f28c5 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:21.899439 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99922895-5ac1-4238-86e1-829a5e5f28c5 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:21.899717 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99922895-5ac1-4238-86e1-829a5e5f28c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:21.920190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-99922895-5ac1-4238-86e1-829a5e5f28c5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:21.921400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-99922895-5ac1-4238-86e1-829a5e5f28c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:21.934134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:21.990705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1157/2302] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:47:21.998505 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb31da-a6a3-40a1-8986-a1481e1dde18 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:21.998859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb31da-a6a3-40a1-8986-a1481e1dde18 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:21.999197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb31da-a6a3-40a1-8986-a1481e1dde18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.020866 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4acb31da-a6a3-40a1-8986-a1481e1dde18 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:22.022073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4acb31da-a6a3-40a1-8986-a1481e1dde18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:22.031091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1146/2303] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:22.035035 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99922895-5ac1-4238-86e1-829a5e5f28c5 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.045013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99922895-5ac1-4238-86e1-829a5e5f28c5 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.049710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35c278-7d27-400f-a6d9-2ebd9f42eb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.050364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35c278-7d27-400f-a6d9-2ebd9f42eb89 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.050648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35c278-7d27-400f-a6d9-2ebd9f42eb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.050923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35c278-7d27-400f-a6d9-2ebd9f42eb89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.051600 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35c278-7d27-400f-a6d9-2ebd9f42eb89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:47:22.051900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35c278-7d27-400f-a6d9-2ebd9f42eb89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.053542 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35c278-7d27-400f-a6d9-2ebd9f42eb89 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.065312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1158/2304] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:47:22.069845 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb31da-a6a3-40a1-8986-a1481e1dde18 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.079783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb31da-a6a3-40a1-8986-a1481e1dde18 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.086393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb7145-b66d-48e5-9f4d-be8bddde1967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.087080 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb7145-b66d-48e5-9f4d-be8bddde1967 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.087533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb7145-b66d-48e5-9f4d-be8bddde1967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.087964 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb7145-b66d-48e5-9f4d-be8bddde1967 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.088648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb7145-b66d-48e5-9f4d-be8bddde1967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.090679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb7145-b66d-48e5-9f4d-be8bddde1967 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.122403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1147/2305] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:22 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:22.126371 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35c278-7d27-400f-a6d9-2ebd9f42eb89 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.137104 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35c278-7d27-400f-a6d9-2ebd9f42eb89 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.201221 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7f3ca-9e3e-4b45-aa98-a2c34245c884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.201858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7f3ca-9e3e-4b45-aa98-a2c34245c884 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.202138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7f3ca-9e3e-4b45-aa98-a2c34245c884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.202411 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7f3ca-9e3e-4b45-aa98-a2c34245c884 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.203323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7f3ca-9e3e-4b45-aa98-a2c34245c884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.205806 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7f3ca-9e3e-4b45-aa98-a2c34245c884 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.226632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1159/2306] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:22 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:22.230706 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb7145-b66d-48e5-9f4d-be8bddde1967 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.241722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb7145-b66d-48e5-9f4d-be8bddde1967 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.246491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadc385-f17e-4060-b330-ea121f68beed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.247102 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadc385-f17e-4060-b330-ea121f68beed admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.247382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadc385-f17e-4060-b330-ea121f68beed admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.247654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadc385-f17e-4060-b330-ea121f68beed admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.248302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dadc385-f17e-4060-b330-ea121f68beed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.250048 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dadc385-f17e-4060-b330-ea121f68beed admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.260004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1148/2307] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:22.264331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7f3ca-9e3e-4b45-aa98-a2c34245c884 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.275558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7f3ca-9e3e-4b45-aa98-a2c34245c884 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.280246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434084e-e727-45cf-a6b3-e33fc7c2dbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.280865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434084e-e727-45cf-a6b3-e33fc7c2dbee admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.281152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434084e-e727-45cf-a6b3-e33fc7c2dbee admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.281421 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434084e-e727-45cf-a6b3-e33fc7c2dbee admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.282060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434084e-e727-45cf-a6b3-e33fc7c2dbee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.284225 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434084e-e727-45cf-a6b3-e33fc7c2dbee admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.296208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1160/2308] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:22.300463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadc385-f17e-4060-b330-ea121f68beed admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.310439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadc385-f17e-4060-b330-ea121f68beed admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.316906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd1e68-96a3-4295-9ad5-5d6a488f00fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.317627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd1e68-96a3-4295-9ad5-5d6a488f00fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.317904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd1e68-96a3-4295-9ad5-5d6a488f00fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.318178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd1e68-96a3-4295-9ad5-5d6a488f00fb admin admin] PATH_INFO: `/v3/projects/2d32dea300d5476faa2d4cfc6cebc1b8/users/443e4a248fe34bedb02d01ccafd4ee06/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.339353 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd1e68-96a3-4295-9ad5-5d6a488f00fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d32dea300d5476faa2d4cfc6cebc1b8, user_id=443e4a248fe34bedb02d01ccafd4ee06, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.342098 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd1e68-96a3-4295-9ad5-5d6a488f00fb admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.357180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1149/2309] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:22 2022] PUT /identity/v3/projects/2d32dea300d5476faa2d4cfc6cebc1b8/users/443e4a248fe34bedb02d01ccafd4ee06/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:22.368661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ddb99-a2f1-49a8-8c47-be766850d679 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.368988 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ddb99-a2f1-49a8-8c47-be766850d679 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.369260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ddb99-a2f1-49a8-8c47-be766850d679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.400603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-603ddb99-a2f1-49a8-8c47-be766850d679 None None] MFA Rules not processed for user `443e4a248fe34bedb02d01ccafd4ee06`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:22.401854 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-603ddb99-a2f1-49a8-8c47-be766850d679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:22.466927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1161/2310] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:22.472107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd1e68-96a3-4295-9ad5-5d6a488f00fb admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.537429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd1e68-96a3-4295-9ad5-5d6a488f00fb admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.542274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0583-f412-4b66-9b31-5869f7e6c91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.542879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0583-f412-4b66-9b31-5869f7e6c91a admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.543181 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0583-f412-4b66-9b31-5869f7e6c91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.543452 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0583-f412-4b66-9b31-5869f7e6c91a admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.544085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0583-f412-4b66-9b31-5869f7e6c91a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.546049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0583-f412-4b66-9b31-5869f7e6c91a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.573466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1150/2311] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:22 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:22.577369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ddb99-a2f1-49a8-8c47-be766850d679 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.588481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ddb99-a2f1-49a8-8c47-be766850d679 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.598126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:22.652549 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4f6a2-2d4f-4dba-b26c-c13465f7c066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.653194 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4f6a2-2d4f-4dba-b26c-c13465f7c066 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.653482 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4f6a2-2d4f-4dba-b26c-c13465f7c066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.653738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4f6a2-2d4f-4dba-b26c-c13465f7c066 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.654364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4f6a2-2d4f-4dba-b26c-c13465f7c066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.656263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4f6a2-2d4f-4dba-b26c-c13465f7c066 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.679695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1162/2312] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:22 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:22.683968 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0583-f412-4b66-9b31-5869f7e6c91a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.695520 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0583-f412-4b66-9b31-5869f7e6c91a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.700251 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8808c98-acd6-4af5-a76d-1f3bddf0a33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.700891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8808c98-acd6-4af5-a76d-1f3bddf0a33b admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.701171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8808c98-acd6-4af5-a76d-1f3bddf0a33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.701438 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8808c98-acd6-4af5-a76d-1f3bddf0a33b admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.702110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8808c98-acd6-4af5-a76d-1f3bddf0a33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.703965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8808c98-acd6-4af5-a76d-1f3bddf0a33b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.714952 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1151/2313] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:22.719353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4f6a2-2d4f-4dba-b26c-c13465f7c066 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.730404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4f6a2-2d4f-4dba-b26c-c13465f7c066 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.735257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a392eb-32f1-41a7-8c9b-f00f446d7ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.735917 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a392eb-32f1-41a7-8c9b-f00f446d7ca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.736196 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a392eb-32f1-41a7-8c9b-f00f446d7ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.736497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a392eb-32f1-41a7-8c9b-f00f446d7ca7 admin admin] PATH_INFO: `/v3/projects/a8a626b1f05548dc93379a9137a1372f/users/93ed2a6549744472a2ff76517f78d5f6/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.751551 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a392eb-32f1-41a7-8c9b-f00f446d7ca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a626b1f05548dc93379a9137a1372f, user_id=93ed2a6549744472a2ff76517f78d5f6, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.754264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a392eb-32f1-41a7-8c9b-f00f446d7ca7 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.769884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1163/2314] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:22 2022] PUT /identity/v3/projects/a8a626b1f05548dc93379a9137a1372f/users/93ed2a6549744472a2ff76517f78d5f6/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:22.774038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8808c98-acd6-4af5-a76d-1f3bddf0a33b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.784550 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8808c98-acd6-4af5-a76d-1f3bddf0a33b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.822567 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:22.843723 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fd3f9-6c3f-4df8-bea5-4d9664be4e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.844335 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd3f9-6c3f-4df8-bea5-4d9664be4e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.844612 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd3f9-6c3f-4df8-bea5-4d9664be4e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.844916 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd3f9-6c3f-4df8-bea5-4d9664be4e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.845558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fd3f9-6c3f-4df8-bea5-4d9664be4e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.847785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fd3f9-6c3f-4df8-bea5-4d9664be4e63 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.857558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1152/2315] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:22.861710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a392eb-32f1-41a7-8c9b-f00f446d7ca7 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:22.872259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a392eb-32f1-41a7-8c9b-f00f446d7ca7 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:22.877030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508d1db-579a-4ee4-8393-33a86cd7e2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:22.877687 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508d1db-579a-4ee4-8393-33a86cd7e2a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.877966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508d1db-579a-4ee4-8393-33a86cd7e2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.878234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508d1db-579a-4ee4-8393-33a86cd7e2a6 admin admin] PATH_INFO: `/v3/projects/a8a626b1f05548dc93379a9137a1372f/users/93ed2a6549744472a2ff76517f78d5f6/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.880134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508d1db-579a-4ee4-8393-33a86cd7e2a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a626b1f05548dc93379a9137a1372f, user_id=93ed2a6549744472a2ff76517f78d5f6, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:22.882832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508d1db-579a-4ee4-8393-33a86cd7e2a6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:22.893706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1164/2316] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:22 2022] PUT /identity/v3/projects/a8a626b1f05548dc93379a9137a1372f/users/93ed2a6549744472a2ff76517f78d5f6/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:47:22.904242 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23417-7a68-4a5e-bdde-c752d5818565 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:22.904565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23417-7a68-4a5e-bdde-c752d5818565 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:22.904865 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23417-7a68-4a5e-bdde-c752d5818565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:22.932897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-bcc23417-7a68-4a5e-bdde-c752d5818565 None None] MFA Rules not processed for user `93ed2a6549744472a2ff76517f78d5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:22.934152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-bcc23417-7a68-4a5e-bdde-c752d5818565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:23.004140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1153/2317] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:23.015558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508d1db-579a-4ee4-8393-33a86cd7e2a6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:23.017320 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6508d1db-579a-4ee4-8393-33a86cd7e2a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:23.075564 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508d1db-579a-4ee4-8393-33a86cd7e2a6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:23.079565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfa2a9-fcdb-44ef-9c40-f1bd70db7330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:23.080698 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa2a9-fcdb-44ef-9c40-f1bd70db7330 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:23.080987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa2a9-fcdb-44ef-9c40-f1bd70db7330 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:23.081260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfa2a9-fcdb-44ef-9c40-f1bd70db7330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:23.090543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfa2a9-fcdb-44ef-9c40-f1bd70db7330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:23.091958 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfa2a9-fcdb-44ef-9c40-f1bd70db7330 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:23.102746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1165/2318] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:23.181840 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23417-7a68-4a5e-bdde-c752d5818565 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:23.184021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-bcc23417-7a68-4a5e-bdde-c752d5818565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:23.197898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23417-7a68-4a5e-bdde-c752d5818565 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:23.201705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a062c15-c546-4343-8c53-d958ef1425b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:23.202314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a062c15-c546-4343-8c53-d958ef1425b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:23.202856 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a062c15-c546-4343-8c53-d958ef1425b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:23.203127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a062c15-c546-4343-8c53-d958ef1425b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:23.260419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a062c15-c546-4343-8c53-d958ef1425b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:23.261822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a062c15-c546-4343-8c53-d958ef1425b3 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:23.271199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1154/2319] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:26.158193 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfa2a9-fcdb-44ef-9c40-f1bd70db7330 service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:26.223863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfa2a9-fcdb-44ef-9c40-f1bd70db7330 service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:26.228354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:26.229020 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:26.229299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:26.229593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] PATH_INFO: `/v3/users/443e4a248fe34bedb02d01ccafd4ee06` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:26.230715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] RBAC: Authorizing `identity:delete_user(user_id=443e4a248fe34bedb02d01ccafd4ee06)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:26.232620 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:26.242834 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:26.280729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443e4a248fe34bedb02d01ccafd4ee06'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:26.293451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443e4a248fe34bedb02d01ccafd4ee06'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:26.294195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443e4a248fe34bedb02d01ccafd4ee06'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:26.301029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443e4a248fe34bedb02d01ccafd4ee06'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:26.301664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443e4a248fe34bedb02d01ccafd4ee06'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:26.314930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1166/2320] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:26 2022] DELETE /identity/v3/users/443e4a248fe34bedb02d01ccafd4ee06 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:26.319459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398e666-69af-4f77-9a35-e2f4498fd2b3 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:26.319998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398e666-69af-4f77-9a35-e2f4498fd2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:26.319998 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398e666-69af-4f77-9a35-e2f4498fd2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:26.339876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-2398e666-69af-4f77-9a35-e2f4498fd2b3 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:26.341114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2398e666-69af-4f77-9a35-e2f4498fd2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:26.397376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:26.406635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1155/2321] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:26.416503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:26.418001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:26.476420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc49d7-2b32-4cbf-b8c7-f9f098b9568b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:26.479941 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05658852-fb2d-4600-819c-bcd9e561e6b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:26.480545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05658852-fb2d-4600-819c-bcd9e561e6b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:26.480821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05658852-fb2d-4600-819c-bcd9e561e6b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:26.481121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05658852-fb2d-4600-819c-bcd9e561e6b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:26.490330 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05658852-fb2d-4600-819c-bcd9e561e6b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:26.491739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05658852-fb2d-4600-819c-bcd9e561e6b2 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:26.500938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1167/2322] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:26.775905 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398e666-69af-4f77-9a35-e2f4498fd2b3 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:26.777838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2398e666-69af-4f77-9a35-e2f4498fd2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:26.791478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398e666-69af-4f77-9a35-e2f4498fd2b3 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:26.796525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:26.797286 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:26.797632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:26.797966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] PATH_INFO: `/v3/projects/2d32dea300d5476faa2d4cfc6cebc1b8` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:26.799673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d32dea300d5476faa2d4cfc6cebc1b8)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:26.801793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:26.852642 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d32dea300d5476faa2d4cfc6cebc1b8', 'request_id': 'req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:26.859114 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d32dea300d5476faa2d4cfc6cebc1b8', 'request_id': 'req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:26.867329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d32dea300d5476faa2d4cfc6cebc1b8', 'request_id': 'req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:26.878989 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] The token cache is being invalidate because project 2d32dea300d5476faa2d4cfc6cebc1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:26.879306 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:26.879931 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:26.881447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1156/2323] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:26 2022] DELETE /identity/v3/projects/2d32dea300d5476faa2d4cfc6cebc1b8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:26.885526 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05658852-fb2d-4600-819c-bcd9e561e6b2 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:26.895328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05658852-fb2d-4600-819c-bcd9e561e6b2 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:26.952943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:26.953578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:26.953861 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:26.954138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] PATH_INFO: `/v3/users/93ed2a6549744472a2ff76517f78d5f6` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:26.955266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ed2a6549744472a2ff76517f78d5f6)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:26.957234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:26.992495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:27.006623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ed2a6549744472a2ff76517f78d5f6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:27.018374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ed2a6549744472a2ff76517f78d5f6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:27.019103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ed2a6549744472a2ff76517f78d5f6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:27.025819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ed2a6549744472a2ff76517f78d5f6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:27.026454 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-a3cacb31-61ca-4356-8412-b8228ceec10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ed2a6549744472a2ff76517f78d5f6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:27.040097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1168/2324] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:26 2022] DELETE /identity/v3/users/93ed2a6549744472a2ff76517f78d5f6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:47:27.566211 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:27.568572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:27.586857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8160f-25cc-4b30-9d85-0ea1e38233e8 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:27.654194 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:27.654748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:27.655028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:27.655302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] PATH_INFO: `/v3/projects/a8a626b1f05548dc93379a9137a1372f` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:27.656416 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a626b1f05548dc93379a9137a1372f)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:27.658364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:27.706618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a626b1f05548dc93379a9137a1372f', 'request_id': 'req-5f601dd9-9a21-49a9-8cca-c4e447962fe2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:27.712979 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a626b1f05548dc93379a9137a1372f', 'request_id': 'req-5f601dd9-9a21-49a9-8cca-c4e447962fe2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:27.720409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a626b1f05548dc93379a9137a1372f', 'request_id': 'req-5f601dd9-9a21-49a9-8cca-c4e447962fe2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:27.731610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] The token cache is being invalidate because project a8a626b1f05548dc93379a9137a1372f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:27.731901 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:27.732521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5f601dd9-9a21-49a9-8cca-c4e447962fe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:27.734031 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1157/2325] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:27 2022] DELETE /identity/v3/projects/a8a626b1f05548dc93379a9137a1372f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:27.738846 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b25fe-33c7-4fba-b597-a3270411b59f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:27.739333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b25fe-33c7-4fba-b597-a3270411b59f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:27.739432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b25fe-33c7-4fba-b597-a3270411b59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:27.761179 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-120b25fe-33c7-4fba-b597-a3270411b59f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:27.762430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-120b25fe-33c7-4fba-b597-a3270411b59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:27.820937 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1169/2326] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:47:27.828811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926c4cf-d5e9-4ab6-8a17-8d7a0e361228 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:27.829128 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926c4cf-d5e9-4ab6-8a17-8d7a0e361228 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:27.829406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926c4cf-d5e9-4ab6-8a17-8d7a0e361228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:27.851506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a926c4cf-d5e9-4ab6-8a17-8d7a0e361228 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:27.852758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a926c4cf-d5e9-4ab6-8a17-8d7a0e361228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:27.860860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1158/2327] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:27.865099 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b25fe-33c7-4fba-b597-a3270411b59f None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:27.875388 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b25fe-33c7-4fba-b597-a3270411b59f None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:27.880535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8e64d-ae5b-448d-8809-494241943925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:27.881170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8e64d-ae5b-448d-8809-494241943925 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:27.881447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8e64d-ae5b-448d-8809-494241943925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:27.881717 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8e64d-ae5b-448d-8809-494241943925 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:27.882357 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8e64d-ae5b-448d-8809-494241943925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:47:27.882685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8e64d-ae5b-448d-8809-494241943925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:27.884272 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8e64d-ae5b-448d-8809-494241943925 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:27.897457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1170/2328] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:47:27.902217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926c4cf-d5e9-4ab6-8a17-8d7a0e361228 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:27.911361 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926c4cf-d5e9-4ab6-8a17-8d7a0e361228 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:27.915895 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57331097-4ddb-4d7f-8841-bf1d46996787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:27.916485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57331097-4ddb-4d7f-8841-bf1d46996787 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:27.916758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57331097-4ddb-4d7f-8841-bf1d46996787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:27.917059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57331097-4ddb-4d7f-8841-bf1d46996787 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:27.917680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57331097-4ddb-4d7f-8841-bf1d46996787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:27.919579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57331097-4ddb-4d7f-8841-bf1d46996787 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:27.924627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:27.956809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1159/2329] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:27 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:27.961142 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8e64d-ae5b-448d-8809-494241943925 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:27.971534 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8e64d-ae5b-448d-8809-494241943925 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.027310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99e20c-a2ab-4757-b885-c2b02376e2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.027910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99e20c-a2ab-4757-b885-c2b02376e2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.028198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99e20c-a2ab-4757-b885-c2b02376e2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.028467 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99e20c-a2ab-4757-b885-c2b02376e2a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.029160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99e20c-a2ab-4757-b885-c2b02376e2a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.031125 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99e20c-a2ab-4757-b885-c2b02376e2a5 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.052654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1171/2330] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:27 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:28.056830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57331097-4ddb-4d7f-8841-bf1d46996787 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:28.069095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57331097-4ddb-4d7f-8841-bf1d46996787 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.074220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58338d2b-f7fc-4697-9694-fcc803d2357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.074829 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58338d2b-f7fc-4697-9694-fcc803d2357a admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.075107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58338d2b-f7fc-4697-9694-fcc803d2357a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.075372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58338d2b-f7fc-4697-9694-fcc803d2357a admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.076007 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58338d2b-f7fc-4697-9694-fcc803d2357a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.077928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58338d2b-f7fc-4697-9694-fcc803d2357a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.088182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1160/2331] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:28.092349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99e20c-a2ab-4757-b885-c2b02376e2a5 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:28.102778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99e20c-a2ab-4757-b885-c2b02376e2a5 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.107528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec118ebe-fdc6-46f9-be5c-54dca8c9c57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.108133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec118ebe-fdc6-46f9-be5c-54dca8c9c57b admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.108414 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec118ebe-fdc6-46f9-be5c-54dca8c9c57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.108819 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec118ebe-fdc6-46f9-be5c-54dca8c9c57b admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.109508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec118ebe-fdc6-46f9-be5c-54dca8c9c57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.111603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec118ebe-fdc6-46f9-be5c-54dca8c9c57b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.123078 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1172/2332] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:28.127725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58338d2b-f7fc-4697-9694-fcc803d2357a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:28.137791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58338d2b-f7fc-4697-9694-fcc803d2357a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.142383 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c72ac0-d701-475b-8956-b4cf447bd56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.143024 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c72ac0-d701-475b-8956-b4cf447bd56e admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.143299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c72ac0-d701-475b-8956-b4cf447bd56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.143566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c72ac0-d701-475b-8956-b4cf447bd56e admin admin] PATH_INFO: `/v3/projects/bb161b8ccaf44642bfa12f034c89b571/users/997056c77373419fa4cae5c02701b064/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.157961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c72ac0-d701-475b-8956-b4cf447bd56e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb161b8ccaf44642bfa12f034c89b571, user_id=997056c77373419fa4cae5c02701b064, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.160668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c72ac0-d701-475b-8956-b4cf447bd56e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.174848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1161/2333] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:28 2022] PUT /identity/v3/projects/bb161b8ccaf44642bfa12f034c89b571/users/997056c77373419fa4cae5c02701b064/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:28.186248 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d3a20-4a16-4def-9773-05ab7e0aab51 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.186657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d3a20-4a16-4def-9773-05ab7e0aab51 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.186938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d3a20-4a16-4def-9773-05ab7e0aab51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.216911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-d15d3a20-4a16-4def-9773-05ab7e0aab51 None None] MFA Rules not processed for user `997056c77373419fa4cae5c02701b064`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:28.218134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d15d3a20-4a16-4def-9773-05ab7e0aab51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:28.222492 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:28.294964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1173/2334] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:28.299951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c72ac0-d701-475b-8956-b4cf447bd56e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:28.363388 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c72ac0-d701-475b-8956-b4cf447bd56e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.368043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29173574-cfd1-4e5c-982d-e50a643355da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.368644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29173574-cfd1-4e5c-982d-e50a643355da admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.368948 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29173574-cfd1-4e5c-982d-e50a643355da admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.369219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29173574-cfd1-4e5c-982d-e50a643355da admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.369850 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29173574-cfd1-4e5c-982d-e50a643355da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.371933 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29173574-cfd1-4e5c-982d-e50a643355da admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.400111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1162/2335] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:28 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:47:28.404132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d3a20-4a16-4def-9773-05ab7e0aab51 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:28.416090 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d3a20-4a16-4def-9773-05ab7e0aab51 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.477369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b2acd-9c3b-4b4b-8cf5-6fb64b0a6294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.477996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b2acd-9c3b-4b4b-8cf5-6fb64b0a6294 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.478279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b2acd-9c3b-4b4b-8cf5-6fb64b0a6294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.478555 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b2acd-9c3b-4b4b-8cf5-6fb64b0a6294 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.479202 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b2acd-9c3b-4b4b-8cf5-6fb64b0a6294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.481364 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b2acd-9c3b-4b4b-8cf5-6fb64b0a6294 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.504046 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1174/2336] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:28 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:28.508140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29173574-cfd1-4e5c-982d-e50a643355da admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:28.518937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29173574-cfd1-4e5c-982d-e50a643355da admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.523650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad8eb4-b876-4d28-a033-8635fc0340f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.524281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad8eb4-b876-4d28-a033-8635fc0340f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.524558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad8eb4-b876-4d28-a033-8635fc0340f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.524826 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad8eb4-b876-4d28-a033-8635fc0340f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.525503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad8eb4-b876-4d28-a033-8635fc0340f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.527105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad8eb4-b876-4d28-a033-8635fc0340f0 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.538140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1163/2337] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:28.542304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b2acd-9c3b-4b4b-8cf5-6fb64b0a6294 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:28.552166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b2acd-9c3b-4b4b-8cf5-6fb64b0a6294 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.556719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3803d-4f00-4b70-98b1-ab181b70291d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.557395 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3803d-4f00-4b70-98b1-ab181b70291d admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.557667 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3803d-4f00-4b70-98b1-ab181b70291d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.557937 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3803d-4f00-4b70-98b1-ab181b70291d admin admin] PATH_INFO: `/v3/projects/e26111ca4e7342b7a2c834f73ec1a9af/users/aed619f2cb454b80a806a7122f5b5c95/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.573798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3803d-4f00-4b70-98b1-ab181b70291d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26111ca4e7342b7a2c834f73ec1a9af, user_id=aed619f2cb454b80a806a7122f5b5c95, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.576404 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3803d-4f00-4b70-98b1-ab181b70291d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.589495 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1175/2338] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:28 2022] PUT /identity/v3/projects/e26111ca4e7342b7a2c834f73ec1a9af/users/aed619f2cb454b80a806a7122f5b5c95/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:47:28.593481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad8eb4-b876-4d28-a033-8635fc0340f0 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:28.605025 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad8eb4-b876-4d28-a033-8635fc0340f0 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.661840 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68d14f-aeeb-44d5-9a58-77f04c819157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.662449 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68d14f-aeeb-44d5-9a58-77f04c819157 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.662725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68d14f-aeeb-44d5-9a58-77f04c819157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.663002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68d14f-aeeb-44d5-9a58-77f04c819157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.663635 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68d14f-aeeb-44d5-9a58-77f04c819157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.665253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68d14f-aeeb-44d5-9a58-77f04c819157 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.675065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1164/2339] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:47:28.679122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3803d-4f00-4b70-98b1-ab181b70291d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:28.690299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3803d-4f00-4b70-98b1-ab181b70291d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.695680 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce11e3-8e74-4267-8b99-3bbc7c372528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.696353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce11e3-8e74-4267-8b99-3bbc7c372528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.696633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce11e3-8e74-4267-8b99-3bbc7c372528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.696944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce11e3-8e74-4267-8b99-3bbc7c372528 admin admin] PATH_INFO: `/v3/projects/e26111ca4e7342b7a2c834f73ec1a9af/users/aed619f2cb454b80a806a7122f5b5c95/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.698904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce11e3-8e74-4267-8b99-3bbc7c372528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26111ca4e7342b7a2c834f73ec1a9af, user_id=aed619f2cb454b80a806a7122f5b5c95, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.701642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce11e3-8e74-4267-8b99-3bbc7c372528 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.713170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1176/2340] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:28 2022] PUT /identity/v3/projects/e26111ca4e7342b7a2c834f73ec1a9af/users/aed619f2cb454b80a806a7122f5b5c95/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:28.723142 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97637409-fa97-4282-9edf-df5184d07ff8 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.723427 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97637409-fa97-4282-9edf-df5184d07ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.723700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97637409-fa97-4282-9edf-df5184d07ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.754109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-97637409-fa97-4282-9edf-df5184d07ff8 None None] MFA Rules not processed for user `aed619f2cb454b80a806a7122f5b5c95`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:28.755330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-97637409-fa97-4282-9edf-df5184d07ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:28.763385 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:28.825182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1165/2341] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:28.837097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce11e3-8e74-4267-8b99-3bbc7c372528 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:28.838904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-02ce11e3-8e74-4267-8b99-3bbc7c372528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:28.897059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce11e3-8e74-4267-8b99-3bbc7c372528 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:28.900729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885ef38-eb72-4a4d-9deb-73a07567ea6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:28.901361 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885ef38-eb72-4a4d-9deb-73a07567ea6e service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:28.901639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885ef38-eb72-4a4d-9deb-73a07567ea6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:28.901914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885ef38-eb72-4a4d-9deb-73a07567ea6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:28.910639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885ef38-eb72-4a4d-9deb-73a07567ea6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:28.912049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885ef38-eb72-4a4d-9deb-73a07567ea6e service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:28.916905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:28.922842 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1177/2342] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:28.989373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97637409-fa97-4282-9edf-df5184d07ff8 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:29.053215 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97637409-fa97-4282-9edf-df5184d07ff8 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:29.057708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:29.058331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:29.058603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:29.058872 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] PATH_INFO: `/v3/users/997056c77373419fa4cae5c02701b064` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:29.059987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=997056c77373419fa4cae5c02701b064)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:29.061924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:29.109501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '997056c77373419fa4cae5c02701b064'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.117486 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '997056c77373419fa4cae5c02701b064'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.118201 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '997056c77373419fa4cae5c02701b064'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.129932 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '997056c77373419fa4cae5c02701b064'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.137177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '997056c77373419fa4cae5c02701b064'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.142983 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1166/2343] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:28 2022] DELETE /identity/v3/users/997056c77373419fa4cae5c02701b064 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:29.148215 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4126f10-9682-4a9f-bfa9-0f3c9fc9cc2e None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:29.148560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4126f10-9682-4a9f-bfa9-0f3c9fc9cc2e None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:29.148878 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4126f10-9682-4a9f-bfa9-0f3c9fc9cc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:29.168508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e4126f10-9682-4a9f-bfa9-0f3c9fc9cc2e None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:29.169742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e4126f10-9682-4a9f-bfa9-0f3c9fc9cc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:29.231952 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1178/2344] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:29.242699 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:29.244131 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:29.275719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:29.308578 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c828e51-6e21-4684-8918-6e927a0cca19 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:29.312278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b34d9-d6ee-411b-bb18-636b80a8031c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:29.312922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b34d9-d6ee-411b-bb18-636b80a8031c service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:29.313204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b34d9-d6ee-411b-bb18-636b80a8031c service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:29.313474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b34d9-d6ee-411b-bb18-636b80a8031c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:29.322679 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b34d9-d6ee-411b-bb18-636b80a8031c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:29.324103 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b34d9-d6ee-411b-bb18-636b80a8031c service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:29.333963 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1167/2345] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:29.606774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4126f10-9682-4a9f-bfa9-0f3c9fc9cc2e None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:29.608243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e4126f10-9682-4a9f-bfa9-0f3c9fc9cc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:29.622921 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4126f10-9682-4a9f-bfa9-0f3c9fc9cc2e None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:29.628235 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:29.629275 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:29.629562 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:29.629832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] PATH_INFO: `/v3/projects/bb161b8ccaf44642bfa12f034c89b571` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:29.631072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb161b8ccaf44642bfa12f034c89b571)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:29.633040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:29.682580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb161b8ccaf44642bfa12f034c89b571', 'request_id': 'req-70a05976-9048-432e-bcf4-2168dd6793b6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.688049 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb161b8ccaf44642bfa12f034c89b571', 'request_id': 'req-70a05976-9048-432e-bcf4-2168dd6793b6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.695519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb161b8ccaf44642bfa12f034c89b571', 'request_id': 'req-70a05976-9048-432e-bcf4-2168dd6793b6'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.707531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] The token cache is being invalidate because project bb161b8ccaf44642bfa12f034c89b571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:29.707842 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.708479 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.710023 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1179/2346] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:29 2022] DELETE /identity/v3/projects/bb161b8ccaf44642bfa12f034c89b571 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:29.713966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b34d9-d6ee-411b-bb18-636b80a8031c service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:29.724113 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b34d9-d6ee-411b-bb18-636b80a8031c service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:29.779109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:29.779661 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:29.779939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:29.780212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] PATH_INFO: `/v3/users/aed619f2cb454b80a806a7122f5b5c95` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:29.781366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed619f2cb454b80a806a7122f5b5c95)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:29.783940 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:29.828355 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed619f2cb454b80a806a7122f5b5c95'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.835670 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed619f2cb454b80a806a7122f5b5c95'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.836323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed619f2cb454b80a806a7122f5b5c95'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.848490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed619f2cb454b80a806a7122f5b5c95'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.855779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-874e38f0-cf8b-4ec2-a893-46c9d96d4db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed619f2cb454b80a806a7122f5b5c95'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:29.861712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1168/2347] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:29 2022] DELETE /identity/v3/users/aed619f2cb454b80a806a7122f5b5c95 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:30.127363 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:30.128790 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:30.141991 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a05976-9048-432e-bcf4-2168dd6793b6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:30.203059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:30.203691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.203967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.204234 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] PATH_INFO: `/v3/projects/e26111ca4e7342b7a2c834f73ec1a9af` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.205948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e26111ca4e7342b7a2c834f73ec1a9af)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:30.207830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:30.237156 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:30.255121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e26111ca4e7342b7a2c834f73ec1a9af', 'request_id': 'req-ddb6096d-9bb3-4578-8412-18f44015b73a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:30.260402 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e26111ca4e7342b7a2c834f73ec1a9af', 'request_id': 'req-ddb6096d-9bb3-4578-8412-18f44015b73a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:30.267143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e26111ca4e7342b7a2c834f73ec1a9af', 'request_id': 'req-ddb6096d-9bb3-4578-8412-18f44015b73a'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:30.278318 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] The token cache is being invalidate because project e26111ca4e7342b7a2c834f73ec1a9af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:30.278608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:30.279293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-ddb6096d-9bb3-4578-8412-18f44015b73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:30.280765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1180/2348] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:30 2022] DELETE /identity/v3/projects/e26111ca4e7342b7a2c834f73ec1a9af => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:30.286799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e9988-315b-4071-8f79-7bc4756b76b7 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.287085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e9988-315b-4071-8f79-7bc4756b76b7 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.287357 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e9988-315b-4071-8f79-7bc4756b76b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.307372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-a88e9988-315b-4071-8f79-7bc4756b76b7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:30.308504 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a88e9988-315b-4071-8f79-7bc4756b76b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:30.349210 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:30.367648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1169/2349] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:47:30.375422 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f900ea-de4d-4006-858b-8543b6d08ea9 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.375788 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f900ea-de4d-4006-858b-8543b6d08ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.376068 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f900ea-de4d-4006-858b-8543b6d08ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.400591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-06f900ea-de4d-4006-858b-8543b6d08ea9 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:30.401841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-06f900ea-de4d-4006-858b-8543b6d08ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:30.409756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1181/2350] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:30.414413 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e9988-315b-4071-8f79-7bc4756b76b7 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:30.424542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e9988-315b-4071-8f79-7bc4756b76b7 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:30.429154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a2bd4-ab7e-4da7-bac2-0492ee3ae767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:30.429823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2bd4-ab7e-4da7-bac2-0492ee3ae767 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.430094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2bd4-ab7e-4da7-bac2-0492ee3ae767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.430489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2bd4-ab7e-4da7-bac2-0492ee3ae767 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.431106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a2bd4-ab7e-4da7-bac2-0492ee3ae767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:47:30.431409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a2bd4-ab7e-4da7-bac2-0492ee3ae767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:30.433033 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a2bd4-ab7e-4da7-bac2-0492ee3ae767 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:30.447238 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1170/2351] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:47:30.451595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f900ea-de4d-4006-858b-8543b6d08ea9 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:30.461038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f900ea-de4d-4006-858b-8543b6d08ea9 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:30.465777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3f6db-fe61-4c24-934d-e7a212ff02ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:30.466378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3f6db-fe61-4c24-934d-e7a212ff02ed admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.466663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3f6db-fe61-4c24-934d-e7a212ff02ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.466939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3f6db-fe61-4c24-934d-e7a212ff02ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.467566 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3f6db-fe61-4c24-934d-e7a212ff02ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:30.469509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3f6db-fe61-4c24-934d-e7a212ff02ed admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:30.496121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1182/2352] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:30 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:47:30.500235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a2bd4-ab7e-4da7-bac2-0492ee3ae767 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:30.510682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a2bd4-ab7e-4da7-bac2-0492ee3ae767 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:30.566702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9121150-219c-404d-b698-8ea3cf346d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:30.567269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9121150-219c-404d-b698-8ea3cf346d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.567532 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9121150-219c-404d-b698-8ea3cf346d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.567798 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9121150-219c-404d-b698-8ea3cf346d53 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.568424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9121150-219c-404d-b698-8ea3cf346d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:30.570369 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9121150-219c-404d-b698-8ea3cf346d53 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:30.594678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1171/2353] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:30 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:47:30.596632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3f6db-fe61-4c24-934d-e7a212ff02ed admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:30.607070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3f6db-fe61-4c24-934d-e7a212ff02ed admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:30.612273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3cf1b-2e71-4ac5-bd44-5fd8a95ddb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:30.612891 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3cf1b-2e71-4ac5-bd44-5fd8a95ddb6c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.613174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3cf1b-2e71-4ac5-bd44-5fd8a95ddb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.613446 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3cf1b-2e71-4ac5-bd44-5fd8a95ddb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.614085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3cf1b-2e71-4ac5-bd44-5fd8a95ddb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:30.615678 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3cf1b-2e71-4ac5-bd44-5fd8a95ddb6c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:30.625459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1183/2354] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:30.629230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9121150-219c-404d-b698-8ea3cf346d53 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:30.639597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9121150-219c-404d-b698-8ea3cf346d53 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:30.644251 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7d11a-f1d2-42d3-a75d-df93d4cd5e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:30.644863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7d11a-f1d2-42d3-a75d-df93d4cd5e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.645148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7d11a-f1d2-42d3-a75d-df93d4cd5e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.645417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7d11a-f1d2-42d3-a75d-df93d4cd5e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.646047 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7d11a-f1d2-42d3-a75d-df93d4cd5e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:30.647627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7d11a-f1d2-42d3-a75d-df93d4cd5e09 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:30.658429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1172/2355] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:47:30.662611 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3cf1b-2e71-4ac5-bd44-5fd8a95ddb6c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:30.672823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3cf1b-2e71-4ac5-bd44-5fd8a95ddb6c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:30.678145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c3bdb-fe37-4638-be35-7f3a8003d6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:30.678793 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c3bdb-fe37-4638-be35-7f3a8003d6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.679079 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c3bdb-fe37-4638-be35-7f3a8003d6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.679353 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c3bdb-fe37-4638-be35-7f3a8003d6c0 admin admin] PATH_INFO: `/v3/projects/105b6d32255744149e71c9e2761c6374/users/9eeb8162c94e43a9a49ca696d2e0797b/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.693677 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c3bdb-fe37-4638-be35-7f3a8003d6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=105b6d32255744149e71c9e2761c6374, user_id=9eeb8162c94e43a9a49ca696d2e0797b, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:30.696328 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c3bdb-fe37-4638-be35-7f3a8003d6c0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:30.710052 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1184/2356] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:30 2022] PUT /identity/v3/projects/105b6d32255744149e71c9e2761c6374/users/9eeb8162c94e43a9a49ca696d2e0797b/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:30.721488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda4c82-07f9-45fe-ba72-496653e403b8 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.721957 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda4c82-07f9-45fe-ba72-496653e403b8 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.722053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda4c82-07f9-45fe-ba72-496653e403b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.749972 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-0dda4c82-07f9-45fe-ba72-496653e403b8 None None] MFA Rules not processed for user `9eeb8162c94e43a9a49ca696d2e0797b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:30.751226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0dda4c82-07f9-45fe-ba72-496653e403b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:30.821071 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1173/2357] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:30.828029 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c3bdb-fe37-4638-be35-7f3a8003d6c0 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:30.829824 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-666c3bdb-fe37-4638-be35-7f3a8003d6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:30.887480 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c3bdb-fe37-4638-be35-7f3a8003d6c0 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:30.891181 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43f1e7-7a9d-4b6e-9fb5-de80b94706e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:30.891792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43f1e7-7a9d-4b6e-9fb5-de80b94706e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:30.892100 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43f1e7-7a9d-4b6e-9fb5-de80b94706e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:30.892378 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43f1e7-7a9d-4b6e-9fb5-de80b94706e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:30.900647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43f1e7-7a9d-4b6e-9fb5-de80b94706e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:30.902119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43f1e7-7a9d-4b6e-9fb5-de80b94706e3 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:30.911520 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1185/2358] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:40.814401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:40.906755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:56.776049 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda4c82-07f9-45fe-ba72-496653e403b8 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:56.837780 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda4c82-07f9-45fe-ba72-496653e403b8 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:56.842272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:56.842887 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:56.843159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:56.843432 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] PATH_INFO: `/v3/users/9eeb8162c94e43a9a49ca696d2e0797b` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:56.844541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eeb8162c94e43a9a49ca696d2e0797b)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:56.846474 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:56.891973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eeb8162c94e43a9a49ca696d2e0797b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:56.899078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eeb8162c94e43a9a49ca696d2e0797b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:56.899744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eeb8162c94e43a9a49ca696d2e0797b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:56.912045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eeb8162c94e43a9a49ca696d2e0797b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:56.919724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eeb8162c94e43a9a49ca696d2e0797b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:56.926393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1174/2359] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:47:56 2022] DELETE /identity/v3/users/9eeb8162c94e43a9a49ca696d2e0797b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:56.930920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8b52e-8484-4579-ad59-3ca299ec0d73 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:56.931419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8b52e-8484-4579-ad59-3ca299ec0d73 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:56.931521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8b52e-8484-4579-ad59-3ca299ec0d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:56.952622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-68a8b52e-8484-4579-ad59-3ca299ec0d73 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:56.953879 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-68a8b52e-8484-4579-ad59-3ca299ec0d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:57.013631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1186/2360] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:57.023591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:57.025029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:57.088419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7dd43-ffa0-4ca8-9ddf-edf78323c88d admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:57.092330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272888ce-a749-48fa-9206-2f614f1919ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:57.092987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272888ce-a749-48fa-9206-2f614f1919ad service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.093271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272888ce-a749-48fa-9206-2f614f1919ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.093613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272888ce-a749-48fa-9206-2f614f1919ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.103663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272888ce-a749-48fa-9206-2f614f1919ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:57.105129 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272888ce-a749-48fa-9206-2f614f1919ad service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:57.116301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1175/2361] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:47:57.383798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8b52e-8484-4579-ad59-3ca299ec0d73 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:57.385372 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-68a8b52e-8484-4579-ad59-3ca299ec0d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:57.399685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8b52e-8484-4579-ad59-3ca299ec0d73 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:57.405205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:57.405847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.406124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.406396 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] PATH_INFO: `/v3/projects/105b6d32255744149e71c9e2761c6374` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.413696 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] RBAC: Authorizing `identity:delete_project(project_id=105b6d32255744149e71c9e2761c6374)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:57.415622 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:57.462991 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '105b6d32255744149e71c9e2761c6374', 'request_id': 'req-aa50c8d1-4af2-4772-8252-51d33e4371af'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:57.468402 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '105b6d32255744149e71c9e2761c6374', 'request_id': 'req-aa50c8d1-4af2-4772-8252-51d33e4371af'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:57.476460 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '105b6d32255744149e71c9e2761c6374', 'request_id': 'req-aa50c8d1-4af2-4772-8252-51d33e4371af'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:57.487527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] The token cache is being invalidate because project 105b6d32255744149e71c9e2761c6374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:47:57.487827 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:57.488455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-aa50c8d1-4af2-4772-8252-51d33e4371af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:47:57.489980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1187/2362] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:47:57 2022] DELETE /identity/v3/projects/105b6d32255744149e71c9e2761c6374 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:57.495274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19410-102f-47c8-a37f-b61f6792c257 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.495560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19410-102f-47c8-a37f-b61f6792c257 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.495830 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19410-102f-47c8-a37f-b61f6792c257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.515078 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-95d19410-102f-47c8-a37f-b61f6792c257 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:57.516272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-95d19410-102f-47c8-a37f-b61f6792c257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:57.532984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:57.577174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1176/2363] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:57.584792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5bb63-3686-4b28-9302-636f96733d94 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.585107 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5bb63-3686-4b28-9302-636f96733d94 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.585380 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5bb63-3686-4b28-9302-636f96733d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.604478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e9e5bb63-3686-4b28-9302-636f96733d94 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:57.605690 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e9e5bb63-3686-4b28-9302-636f96733d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:57.614084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1188/2364] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:57.618350 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19410-102f-47c8-a37f-b61f6792c257 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:57.627978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19410-102f-47c8-a37f-b61f6792c257 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:57.633077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862e450-8306-4381-b5f1-c14d85cd5ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:57.633674 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862e450-8306-4381-b5f1-c14d85cd5ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.633950 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862e450-8306-4381-b5f1-c14d85cd5ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.634222 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862e450-8306-4381-b5f1-c14d85cd5ff6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.634864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862e450-8306-4381-b5f1-c14d85cd5ff6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:47:57.635166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862e450-8306-4381-b5f1-c14d85cd5ff6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:57.636759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862e450-8306-4381-b5f1-c14d85cd5ff6 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:57.647995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1177/2365] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:47:57.652040 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5bb63-3686-4b28-9302-636f96733d94 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:57.661148 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5bb63-3686-4b28-9302-636f96733d94 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:57.665857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06a1e8-9493-4b71-aeb1-d2fe850cce3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:57.666453 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06a1e8-9493-4b71-aeb1-d2fe850cce3c admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.666759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06a1e8-9493-4b71-aeb1-d2fe850cce3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.667031 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06a1e8-9493-4b71-aeb1-d2fe850cce3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.667657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06a1e8-9493-4b71-aeb1-d2fe850cce3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:57.669614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06a1e8-9493-4b71-aeb1-d2fe850cce3c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:57.697096 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1189/2366] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:57 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:57.701770 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862e450-8306-4381-b5f1-c14d85cd5ff6 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:57.711911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862e450-8306-4381-b5f1-c14d85cd5ff6 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:57.768047 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4399bb-039b-4a15-96a5-eaf6368fe2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:57.768573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4399bb-039b-4a15-96a5-eaf6368fe2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.768867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4399bb-039b-4a15-96a5-eaf6368fe2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.769173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4399bb-039b-4a15-96a5-eaf6368fe2b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.769779 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4399bb-039b-4a15-96a5-eaf6368fe2b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:57.771695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4399bb-039b-4a15-96a5-eaf6368fe2b8 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:57.792936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1178/2367] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:57 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:47:57.797232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06a1e8-9493-4b71-aeb1-d2fe850cce3c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:57.808171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06a1e8-9493-4b71-aeb1-d2fe850cce3c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:57.812673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f7d4e-550b-48aa-8e62-591e5d2417cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:57.813293 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f7d4e-550b-48aa-8e62-591e5d2417cc admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.813569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f7d4e-550b-48aa-8e62-591e5d2417cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.813838 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f7d4e-550b-48aa-8e62-591e5d2417cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.814466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f7d4e-550b-48aa-8e62-591e5d2417cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:57.816650 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f7d4e-550b-48aa-8e62-591e5d2417cc admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:57.826491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1190/2368] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:57.830509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4399bb-039b-4a15-96a5-eaf6368fe2b8 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:57.840262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4399bb-039b-4a15-96a5-eaf6368fe2b8 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:57.844825 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fc4f5-fe5c-48f5-9065-c67b738df2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:57.845442 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fc4f5-fe5c-48f5-9065-c67b738df2ac admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.845714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fc4f5-fe5c-48f5-9065-c67b738df2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.845981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fc4f5-fe5c-48f5-9065-c67b738df2ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.846608 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fc4f5-fe5c-48f5-9065-c67b738df2ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:57.848208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fc4f5-fe5c-48f5-9065-c67b738df2ac admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:57.858546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1179/2369] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:47:57.864978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f7d4e-550b-48aa-8e62-591e5d2417cc admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:57.874799 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f7d4e-550b-48aa-8e62-591e5d2417cc admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:57.879951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e96cfb-970e-495a-ab47-8c09dbead9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:57.880590 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e96cfb-970e-495a-ab47-8c09dbead9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.880890 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e96cfb-970e-495a-ab47-8c09dbead9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.881166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e96cfb-970e-495a-ab47-8c09dbead9f3 admin admin] PATH_INFO: `/v3/projects/7dbb688bac7a419b9bf617260a6e8e1e/users/b9bcc4cd9dc34c8780436b59b9b3c957/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.896332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e96cfb-970e-495a-ab47-8c09dbead9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dbb688bac7a419b9bf617260a6e8e1e, user_id=b9bcc4cd9dc34c8780436b59b9b3c957, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:57.899000 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e96cfb-970e-495a-ab47-8c09dbead9f3 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:57.911829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1191/2370] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:57 2022] PUT /identity/v3/projects/7dbb688bac7a419b9bf617260a6e8e1e/users/b9bcc4cd9dc34c8780436b59b9b3c957/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:57.922999 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd96a0-bd7c-4373-9ca7-a959eff255ab None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:57.923284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd96a0-bd7c-4373-9ca7-a959eff255ab None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:57.923555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd96a0-bd7c-4373-9ca7-a959eff255ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:57.951767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-65dd96a0-bd7c-4373-9ca7-a959eff255ab None None] MFA Rules not processed for user `b9bcc4cd9dc34c8780436b59b9b3c957`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:57.952996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-65dd96a0-bd7c-4373-9ca7-a959eff255ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:58.015377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1180/2371] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:58.020050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e96cfb-970e-495a-ab47-8c09dbead9f3 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.040943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:58.082817 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e96cfb-970e-495a-ab47-8c09dbead9f3 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.087511 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2c96-09f0-4508-91a1-ee43bef7883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.088109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2c96-09f0-4508-91a1-ee43bef7883b admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.088386 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2c96-09f0-4508-91a1-ee43bef7883b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.088657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2c96-09f0-4508-91a1-ee43bef7883b admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.089312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2c96-09f0-4508-91a1-ee43bef7883b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.091197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2c96-09f0-4508-91a1-ee43bef7883b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.117277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1192/2372] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:58 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:58.121365 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd96a0-bd7c-4373-9ca7-a959eff255ab None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.132140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd96a0-bd7c-4373-9ca7-a959eff255ab None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.144708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:58.198900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de77c30-ebe2-41d8-96ae-25a6864f61eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.199513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de77c30-ebe2-41d8-96ae-25a6864f61eb admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.199791 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de77c30-ebe2-41d8-96ae-25a6864f61eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.200062 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de77c30-ebe2-41d8-96ae-25a6864f61eb admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.200681 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de77c30-ebe2-41d8-96ae-25a6864f61eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.203183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de77c30-ebe2-41d8-96ae-25a6864f61eb admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.236502 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1181/2373] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:58 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:58.241162 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2c96-09f0-4508-91a1-ee43bef7883b admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.253006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2c96-09f0-4508-91a1-ee43bef7883b admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.259059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642909c3-3400-4305-a375-57aa9d062124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.259674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642909c3-3400-4305-a375-57aa9d062124 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.259941 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642909c3-3400-4305-a375-57aa9d062124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.260210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642909c3-3400-4305-a375-57aa9d062124 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.260887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642909c3-3400-4305-a375-57aa9d062124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.262473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642909c3-3400-4305-a375-57aa9d062124 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.275276 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1193/2374] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:58.285244 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de77c30-ebe2-41d8-96ae-25a6864f61eb admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.295337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de77c30-ebe2-41d8-96ae-25a6864f61eb admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.300276 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d00f7-a508-4ec8-9db9-bc838b4fba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.300907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d00f7-a508-4ec8-9db9-bc838b4fba7d admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.301191 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d00f7-a508-4ec8-9db9-bc838b4fba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.301602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d00f7-a508-4ec8-9db9-bc838b4fba7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.302246 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d00f7-a508-4ec8-9db9-bc838b4fba7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.303839 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d00f7-a508-4ec8-9db9-bc838b4fba7d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.313684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1182/2375] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:58.317778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642909c3-3400-4305-a375-57aa9d062124 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.328066 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642909c3-3400-4305-a375-57aa9d062124 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.333258 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d8ade-6027-4e45-b277-d39e1fbda44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.333907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d8ade-6027-4e45-b277-d39e1fbda44e admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.334185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d8ade-6027-4e45-b277-d39e1fbda44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.334457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d8ade-6027-4e45-b277-d39e1fbda44e admin admin] PATH_INFO: `/v3/projects/1060fa71a3a747a4b9907433bb78fe1f/users/26f8d2551acb4c7c8d42501e3a526d21/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.353051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d8ade-6027-4e45-b277-d39e1fbda44e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1060fa71a3a747a4b9907433bb78fe1f, user_id=26f8d2551acb4c7c8d42501e3a526d21, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.355734 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d8ade-6027-4e45-b277-d39e1fbda44e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.368448 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1194/2376] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:58 2022] PUT /identity/v3/projects/1060fa71a3a747a4b9907433bb78fe1f/users/26f8d2551acb4c7c8d42501e3a526d21/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:47:58.378841 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7b094-dea5-4dbf-8f46-c60193daf158 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.379131 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7b094-dea5-4dbf-8f46-c60193daf158 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.379407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7b094-dea5-4dbf-8f46-c60193daf158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.407774 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d9c7b094-dea5-4dbf-8f46-c60193daf158 None None] MFA Rules not processed for user `26f8d2551acb4c7c8d42501e3a526d21`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:58.409023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d9c7b094-dea5-4dbf-8f46-c60193daf158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:58.477707 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1183/2377] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:58.482996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d8ade-6027-4e45-b277-d39e1fbda44e admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.543895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d8ade-6027-4e45-b277-d39e1fbda44e admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.548643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965845b-3aac-433e-bf2c-0b1844913a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.549273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965845b-3aac-433e-bf2c-0b1844913a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.549551 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965845b-3aac-433e-bf2c-0b1844913a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.549844 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965845b-3aac-433e-bf2c-0b1844913a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.550476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965845b-3aac-433e-bf2c-0b1844913a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.552362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965845b-3aac-433e-bf2c-0b1844913a51 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.578108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1195/2378] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:47:58 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:58.582727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7b094-dea5-4dbf-8f46-c60193daf158 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.593549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7b094-dea5-4dbf-8f46-c60193daf158 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.648962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6800d85-16c2-4cbe-916f-ce8308fe0f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.649564 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6800d85-16c2-4cbe-916f-ce8308fe0f5d admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.649844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6800d85-16c2-4cbe-916f-ce8308fe0f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.650117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6800d85-16c2-4cbe-916f-ce8308fe0f5d admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.650778 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6800d85-16c2-4cbe-916f-ce8308fe0f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.652712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6800d85-16c2-4cbe-916f-ce8308fe0f5d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.673828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1184/2379] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:47:58 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:47:58.677912 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965845b-3aac-433e-bf2c-0b1844913a51 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.688614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965845b-3aac-433e-bf2c-0b1844913a51 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.693755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902ceff-2001-4c08-bc1d-5486fcabb1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.694339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902ceff-2001-4c08-bc1d-5486fcabb1ad admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.694614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902ceff-2001-4c08-bc1d-5486fcabb1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.694883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902ceff-2001-4c08-bc1d-5486fcabb1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.695539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902ceff-2001-4c08-bc1d-5486fcabb1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.697277 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902ceff-2001-4c08-bc1d-5486fcabb1ad admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.708277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6800d85-16c2-4cbe-916f-ce8308fe0f5d admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.710068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d6800d85-16c2-4cbe-916f-ce8308fe0f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:58.710218 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1196/2380] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:47:58.714057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902ceff-2001-4c08-bc1d-5486fcabb1ad admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.723537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902ceff-2001-4c08-bc1d-5486fcabb1ad admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.728198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab9b47-6848-4eea-a4be-0ac44aadf62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.728862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9b47-6848-4eea-a4be-0ac44aadf62d admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.729143 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9b47-6848-4eea-a4be-0ac44aadf62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.729418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9b47-6848-4eea-a4be-0ac44aadf62d admin admin] PATH_INFO: `/v3/projects/76fdfe87be3e4d209731e1a4648daf87/users/16a43f2063254dc880d3ebba6d090c30/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.732331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:58.745125 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab9b47-6848-4eea-a4be-0ac44aadf62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76fdfe87be3e4d209731e1a4648daf87, user_id=16a43f2063254dc880d3ebba6d090c30, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.747899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab9b47-6848-4eea-a4be-0ac44aadf62d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.761956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1197/2381] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:58 2022] PUT /identity/v3/projects/76fdfe87be3e4d209731e1a4648daf87/users/16a43f2063254dc880d3ebba6d090c30/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:47:58.766006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab9b47-6848-4eea-a4be-0ac44aadf62d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.774393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6800d85-16c2-4cbe-916f-ce8308fe0f5d admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.786964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:58.822986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb3477-073e-4e3a-9e3f-d9eca11199ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.823512 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb3477-073e-4e3a-9e3f-d9eca11199ee service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.823822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb3477-073e-4e3a-9e3f-d9eca11199ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.824094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb3477-073e-4e3a-9e3f-d9eca11199ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.825939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-83eb3477-073e-4e3a-9e3f-d9eca11199ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:58.829038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab9b47-6848-4eea-a4be-0ac44aadf62d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.834333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c3771-bdc2-4f4b-9f2d-217aa7f16861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.834933 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3771-bdc2-4f4b-9f2d-217aa7f16861 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.835213 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3771-bdc2-4f4b-9f2d-217aa7f16861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.835508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3771-bdc2-4f4b-9f2d-217aa7f16861 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.836178 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c3771-bdc2-4f4b-9f2d-217aa7f16861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.837915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c3771-bdc2-4f4b-9f2d-217aa7f16861 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.847743 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1198/2382] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:47:58.852141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c3771-bdc2-4f4b-9f2d-217aa7f16861 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:58.861758 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c3771-bdc2-4f4b-9f2d-217aa7f16861 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:58.866610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c5139-ed56-40af-ae88-7326dc7e60c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:58.867268 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c5139-ed56-40af-ae88-7326dc7e60c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.867663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c5139-ed56-40af-ae88-7326dc7e60c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.867942 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c5139-ed56-40af-ae88-7326dc7e60c4 admin admin] PATH_INFO: `/v3/projects/76fdfe87be3e4d209731e1a4648daf87/users/16a43f2063254dc880d3ebba6d090c30/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.869885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c5139-ed56-40af-ae88-7326dc7e60c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76fdfe87be3e4d209731e1a4648daf87, user_id=16a43f2063254dc880d3ebba6d090c30, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.872559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c5139-ed56-40af-ae88-7326dc7e60c4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.885032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1199/2383] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:47:58 2022] PUT /identity/v3/projects/76fdfe87be3e4d209731e1a4648daf87/users/16a43f2063254dc880d3ebba6d090c30/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:47:58.886620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb3477-073e-4e3a-9e3f-d9eca11199ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:58.888068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb3477-073e-4e3a-9e3f-d9eca11199ee service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:58.895317 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387e62a-ed60-403b-8326-fbda341823ae None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:58.895607 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387e62a-ed60-403b-8326-fbda341823ae None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:58.895887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387e62a-ed60-403b-8326-fbda341823ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:58.897813 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1185/2384] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:47:58.924294 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-9387e62a-ed60-403b-8326-fbda341823ae None None] MFA Rules not processed for user `16a43f2063254dc880d3ebba6d090c30`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:47:58.925584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9387e62a-ed60-403b-8326-fbda341823ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:58.993491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1200/2385] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:47:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:47:59.005603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb3477-073e-4e3a-9e3f-d9eca11199ee service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:47:59.006993 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-83eb3477-073e-4e3a-9e3f-d9eca11199ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:47:59.064138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb3477-073e-4e3a-9e3f-d9eca11199ee service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:47:59.103651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:47:59.112804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62e710-ca1d-4297-8e87-1e816367524e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:47:59.113439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62e710-ca1d-4297-8e87-1e816367524e service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:47:59.113744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62e710-ca1d-4297-8e87-1e816367524e service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:47:59.114009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62e710-ca1d-4297-8e87-1e816367524e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:47:59.172820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62e710-ca1d-4297-8e87-1e816367524e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:47:59.174581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62e710-ca1d-4297-8e87-1e816367524e service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:47:59.184978 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1186/2386] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:47:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:00.779246 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387e62a-ed60-403b-8326-fbda341823ae None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:00.781152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9387e62a-ed60-403b-8326-fbda341823ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:00.798006 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387e62a-ed60-403b-8326-fbda341823ae None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:00.802397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8589926-4215-4043-8d61-a8046ff2b968 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:00.803285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8589926-4215-4043-8d61-a8046ff2b968 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:00.803661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8589926-4215-4043-8d61-a8046ff2b968 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:00.804400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8589926-4215-4043-8d61-a8046ff2b968 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:00.884433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8589926-4215-4043-8d61-a8046ff2b968 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:00.886306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8589926-4215-4043-8d61-a8046ff2b968 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:00.898261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1201/2387] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:01.485156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62e710-ca1d-4297-8e87-1e816367524e service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:01.494594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62e710-ca1d-4297-8e87-1e816367524e service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:01.498265 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b71db-2112-4d39-9325-c48773014761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:01.498859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b71db-2112-4d39-9325-c48773014761 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:01.499135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b71db-2112-4d39-9325-c48773014761 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:01.499406 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b71db-2112-4d39-9325-c48773014761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:01.508334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b71db-2112-4d39-9325-c48773014761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:01.509861 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b71db-2112-4d39-9325-c48773014761 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:01.519112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1187/2388] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:01.657839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8589926-4215-4043-8d61-a8046ff2b968 service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:01.659231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d8589926-4215-4043-8d61-a8046ff2b968 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:01.690610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:01.728656 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8589926-4215-4043-8d61-a8046ff2b968 service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:01.733602 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:01.734201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:01.734563 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:01.734755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] PATH_INFO: `/v3/users/e4ed95feeb734b609373e251e9796bdc` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:01.735910 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ed95feeb734b609373e251e9796bdc)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:01.738485 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:01.786092 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ed95feeb734b609373e251e9796bdc'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:01.797970 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ed95feeb734b609373e251e9796bdc'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:01.798592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ed95feeb734b609373e251e9796bdc'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:01.805420 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ed95feeb734b609373e251e9796bdc'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:01.806050 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ed95feeb734b609373e251e9796bdc'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:01.818405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1202/2389] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:48:01 2022] DELETE /identity/v3/users/e4ed95feeb734b609373e251e9796bdc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:01.822931 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491b041-6640-4033-8bc3-6df2ee23e3ee None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:01.823217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491b041-6640-4033-8bc3-6df2ee23e3ee None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:01.823488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491b041-6640-4033-8bc3-6df2ee23e3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:01.842876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9491b041-6640-4033-8bc3-6df2ee23e3ee None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:01.844068 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9491b041-6640-4033-8bc3-6df2ee23e3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:01.901380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1188/2390] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:01.911730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:01.913189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:01.971956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bac55-8f57-4ebe-8cd4-ae6f61bf6061 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:01.975661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9baf6c-f274-4239-a59d-9659b3e9259b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:01.976263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9baf6c-f274-4239-a59d-9659b3e9259b service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:01.976544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9baf6c-f274-4239-a59d-9659b3e9259b service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:01.976815 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9baf6c-f274-4239-a59d-9659b3e9259b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:01.986842 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9baf6c-f274-4239-a59d-9659b3e9259b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:01.988786 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9baf6c-f274-4239-a59d-9659b3e9259b service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:01.998164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1203/2391] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:02.256194 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491b041-6640-4033-8bc3-6df2ee23e3ee None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:02.257733 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9491b041-6640-4033-8bc3-6df2ee23e3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:02.276615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491b041-6640-4033-8bc3-6df2ee23e3ee None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:02.281312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:02.281956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:02.282234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:02.282507 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] PATH_INFO: `/v3/projects/30577d476e034fb39450ad9ee6a0e4ed` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:02.283684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30577d476e034fb39450ad9ee6a0e4ed)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:02.285668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:02.337302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30577d476e034fb39450ad9ee6a0e4ed', 'request_id': 'req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:02.343000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30577d476e034fb39450ad9ee6a0e4ed', 'request_id': 'req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:02.350329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30577d476e034fb39450ad9ee6a0e4ed', 'request_id': 'req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:02.363208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] The token cache is being invalidate because project 30577d476e034fb39450ad9ee6a0e4ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:48:02.363208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:02.364199 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-a42ff7ee-4af1-44bd-8191-7ea8e28284f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:02.366719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1189/2392] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:48:02 2022] DELETE /identity/v3/projects/30577d476e034fb39450ad9ee6a0e4ed => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:02.371736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d8db7-b13c-4790-aa9e-e81ae73e9558 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:02.371922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d8db7-b13c-4790-aa9e-e81ae73e9558 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:02.372198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d8db7-b13c-4790-aa9e-e81ae73e9558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:02.393285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-670d8db7-b13c-4790-aa9e-e81ae73e9558 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:02.395170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-670d8db7-b13c-4790-aa9e-e81ae73e9558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:02.439671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:02.457361 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1204/2393] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:02.466198 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495eb489-09f7-4206-92c2-502cc7a0ce46 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:02.466485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495eb489-09f7-4206-92c2-502cc7a0ce46 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:02.466762 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495eb489-09f7-4206-92c2-502cc7a0ce46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:02.487814 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:02.493860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-495eb489-09f7-4206-92c2-502cc7a0ce46 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:02.495844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-495eb489-09f7-4206-92c2-502cc7a0ce46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:02.506875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1190/2394] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:02.517336 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d8db7-b13c-4790-aa9e-e81ae73e9558 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:02.527684 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d8db7-b13c-4790-aa9e-e81ae73e9558 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:02.532650 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adccd3e-7234-4b80-bd53-988adfc43d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:02.533298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adccd3e-7234-4b80-bd53-988adfc43d68 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:02.533582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adccd3e-7234-4b80-bd53-988adfc43d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:02.533853 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adccd3e-7234-4b80-bd53-988adfc43d68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:02.535109 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adccd3e-7234-4b80-bd53-988adfc43d68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:48:02.535419 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adccd3e-7234-4b80-bd53-988adfc43d68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:02.537038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adccd3e-7234-4b80-bd53-988adfc43d68 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:02.554843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1205/2395] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:48:02.556292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495eb489-09f7-4206-92c2-502cc7a0ce46 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:02.566182 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495eb489-09f7-4206-92c2-502cc7a0ce46 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:02.570790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fff28-1369-4143-83c0-4bcbdcdd8af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:02.571392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fff28-1369-4143-83c0-4bcbdcdd8af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:02.571667 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fff28-1369-4143-83c0-4bcbdcdd8af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:02.571936 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fff28-1369-4143-83c0-4bcbdcdd8af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:02.572570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fff28-1369-4143-83c0-4bcbdcdd8af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:02.574494 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fff28-1369-4143-83c0-4bcbdcdd8af3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:02.603695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1191/2396] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:48:02 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:48:02.607835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adccd3e-7234-4b80-bd53-988adfc43d68 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:02.625019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adccd3e-7234-4b80-bd53-988adfc43d68 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:02.677972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005e635-51dd-44d5-9b4b-11336c70f908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:02.678582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005e635-51dd-44d5-9b4b-11336c70f908 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:02.678863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005e635-51dd-44d5-9b4b-11336c70f908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:02.679156 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005e635-51dd-44d5-9b4b-11336c70f908 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:02.679795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005e635-51dd-44d5-9b4b-11336c70f908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:02.681857 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005e635-51dd-44d5-9b4b-11336c70f908 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:02.707619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1206/2397] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:48:02 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:48:02.708851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fff28-1369-4143-83c0-4bcbdcdd8af3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:02.719063 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fff28-1369-4143-83c0-4bcbdcdd8af3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:02.724579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e794ae-ebd9-4c19-97e9-ce1e1f43e7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:02.725214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e794ae-ebd9-4c19-97e9-ce1e1f43e7bd admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:02.725493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e794ae-ebd9-4c19-97e9-ce1e1f43e7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:02.725764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e794ae-ebd9-4c19-97e9-ce1e1f43e7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:02.726409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e794ae-ebd9-4c19-97e9-ce1e1f43e7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:02.728003 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e794ae-ebd9-4c19-97e9-ce1e1f43e7bd admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:02.741184 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1192/2398] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:02.742342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005e635-51dd-44d5-9b4b-11336c70f908 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:02.752201 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005e635-51dd-44d5-9b4b-11336c70f908 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:02.756777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbf7eb-b695-4dfb-b4dd-880be140baac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:02.757433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbf7eb-b695-4dfb-b4dd-880be140baac admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:02.757715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbf7eb-b695-4dfb-b4dd-880be140baac admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:02.757983 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbf7eb-b695-4dfb-b4dd-880be140baac admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:02.758614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbf7eb-b695-4dfb-b4dd-880be140baac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:02.760230 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbf7eb-b695-4dfb-b4dd-880be140baac admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:02.771177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1207/2399] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:02.775411 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e794ae-ebd9-4c19-97e9-ce1e1f43e7bd admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:02.787201 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e794ae-ebd9-4c19-97e9-ce1e1f43e7bd admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:02.791869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0f1bc-fa47-4d0d-8887-6b0dd196f14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:02.792816 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0f1bc-fa47-4d0d-8887-6b0dd196f14a admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:02.793138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0f1bc-fa47-4d0d-8887-6b0dd196f14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:02.793419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0f1bc-fa47-4d0d-8887-6b0dd196f14a admin admin] PATH_INFO: `/v3/projects/670892b833e0414eb6d4b639d85f98c6/users/6221c883d64b43349bf01523e5921384/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:02.810707 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0f1bc-fa47-4d0d-8887-6b0dd196f14a admin admin] RBAC: Authorizing `identity:create_grant(project_id=670892b833e0414eb6d4b639d85f98c6, user_id=6221c883d64b43349bf01523e5921384, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:02.813690 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0f1bc-fa47-4d0d-8887-6b0dd196f14a admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:02.830175 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1193/2400] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:48:02 2022] PUT /identity/v3/projects/670892b833e0414eb6d4b639d85f98c6/users/6221c883d64b43349bf01523e5921384/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:48:02.841514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a762d1-c7c6-4f61-9360-1ad74000e898 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:02.841979 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a762d1-c7c6-4f61-9360-1ad74000e898 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:02.842064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a762d1-c7c6-4f61-9360-1ad74000e898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:02.876431 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-33a762d1-c7c6-4f61-9360-1ad74000e898 None None] MFA Rules not processed for user `6221c883d64b43349bf01523e5921384`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:02.877531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-33a762d1-c7c6-4f61-9360-1ad74000e898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:03.017645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1208/2401] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:03.030343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0f1bc-fa47-4d0d-8887-6b0dd196f14a admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:03.032412 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d9f0f1bc-fa47-4d0d-8887-6b0dd196f14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:03.101294 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0f1bc-fa47-4d0d-8887-6b0dd196f14a admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:03.107263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e9132-2e27-4bf8-a8b5-d193e8a3aa72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:03.108186 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e9132-2e27-4bf8-a8b5-d193e8a3aa72 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:03.108531 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e9132-2e27-4bf8-a8b5-d193e8a3aa72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:03.108935 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e9132-2e27-4bf8-a8b5-d193e8a3aa72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:03.119048 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e9132-2e27-4bf8-a8b5-d193e8a3aa72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:03.120513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e9132-2e27-4bf8-a8b5-d193e8a3aa72 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:03.130889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1194/2402] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:48:05.090365 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a762d1-c7c6-4f61-9360-1ad74000e898 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:05.092374 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-33a762d1-c7c6-4f61-9360-1ad74000e898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:05.152832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:05.167576 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a762d1-c7c6-4f61-9360-1ad74000e898 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:05.171966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d031c-5eb5-417f-81fb-429c80d33109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:05.172663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d031c-5eb5-417f-81fb-429c80d33109 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:05.172976 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d031c-5eb5-417f-81fb-429c80d33109 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:05.173310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d031c-5eb5-417f-81fb-429c80d33109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:05.175476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ae3d031c-5eb5-417f-81fb-429c80d33109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:05.298130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d031c-5eb5-417f-81fb-429c80d33109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:05.300228 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d031c-5eb5-417f-81fb-429c80d33109 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:05.311310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1209/2403] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:05.606965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e9132-2e27-4bf8-a8b5-d193e8a3aa72 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:05.608509 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-a14e9132-2e27-4bf8-a8b5-d193e8a3aa72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:05.623642 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e9132-2e27-4bf8-a8b5-d193e8a3aa72 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:05.628934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:05.629633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:05.629982 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:05.630323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] PATH_INFO: `/v3/users/b9bcc4cd9dc34c8780436b59b9b3c957` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:05.631869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9bcc4cd9dc34c8780436b59b9b3c957)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:05.633598 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:05.680848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9bcc4cd9dc34c8780436b59b9b3c957'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:05.688115 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9bcc4cd9dc34c8780436b59b9b3c957'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:05.688799 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9bcc4cd9dc34c8780436b59b9b3c957'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:05.702333 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9bcc4cd9dc34c8780436b59b9b3c957'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:05.710109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9bcc4cd9dc34c8780436b59b9b3c957'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:05.717274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1195/2404] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:48:05 2022] DELETE /identity/v3/users/b9bcc4cd9dc34c8780436b59b9b3c957 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:05.721840 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb02e75-0c7b-4f4b-99bc-adcf63d5c11a None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:05.722243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb02e75-0c7b-4f4b-99bc-adcf63d5c11a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:05.722741 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb02e75-0c7b-4f4b-99bc-adcf63d5c11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:05.742540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-ebb02e75-0c7b-4f4b-99bc-adcf63d5c11a None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:05.743756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ebb02e75-0c7b-4f4b-99bc-adcf63d5c11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:05.803322 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1210/2405] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:05.813332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:05.814766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:05.839666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:05.887064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11112c-2ad9-4a5b-b703-b99e787d99a7 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:05.890759 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56804-252f-4ce5-971b-ce33ad973892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:05.891373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56804-252f-4ce5-971b-ce33ad973892 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:05.891651 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56804-252f-4ce5-971b-ce33ad973892 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:05.891926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56804-252f-4ce5-971b-ce33ad973892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:05.901423 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b56804-252f-4ce5-971b-ce33ad973892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:05.902890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b56804-252f-4ce5-971b-ce33ad973892 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:05.912945 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1196/2406] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:06.275934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb02e75-0c7b-4f4b-99bc-adcf63d5c11a None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:06.277510 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ebb02e75-0c7b-4f4b-99bc-adcf63d5c11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:06.311130 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb02e75-0c7b-4f4b-99bc-adcf63d5c11a None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:06.325097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:06.325755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:06.326298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:06.333494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] PATH_INFO: `/v3/projects/7dbb688bac7a419b9bf617260a6e8e1e` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:06.334829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dbb688bac7a419b9bf617260a6e8e1e)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:06.336757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:06.462313 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dbb688bac7a419b9bf617260a6e8e1e', 'request_id': 'req-396de91c-58a1-42d1-9385-04ea120b9b80'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:06.477118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dbb688bac7a419b9bf617260a6e8e1e', 'request_id': 'req-396de91c-58a1-42d1-9385-04ea120b9b80'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:06.484665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dbb688bac7a419b9bf617260a6e8e1e', 'request_id': 'req-396de91c-58a1-42d1-9385-04ea120b9b80'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:06.516582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] The token cache is being invalidate because project 7dbb688bac7a419b9bf617260a6e8e1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:48:06.516986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:06.517978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:06.520283 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1211/2407] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:48:06 2022] DELETE /identity/v3/projects/7dbb688bac7a419b9bf617260a6e8e1e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:06.530001 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56804-252f-4ce5-971b-ce33ad973892 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:06.538070 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56804-252f-4ce5-971b-ce33ad973892 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:06.647157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:06.648001 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:06.648419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:06.648783 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] PATH_INFO: `/v3/users/26f8d2551acb4c7c8d42501e3a526d21` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:06.650043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] RBAC: Authorizing `identity:delete_user(user_id=26f8d2551acb4c7c8d42501e3a526d21)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:06.652229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:06.721269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26f8d2551acb4c7c8d42501e3a526d21'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:06.733607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26f8d2551acb4c7c8d42501e3a526d21'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:06.736676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26f8d2551acb4c7c8d42501e3a526d21'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:06.757591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26f8d2551acb4c7c8d42501e3a526d21'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:06.774961 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26f8d2551acb4c7c8d42501e3a526d21'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:06.783696 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1197/2408] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:48:06 2022] DELETE /identity/v3/users/26f8d2551acb4c7c8d42501e3a526d21 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:07.250653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:07.252579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:07.275158 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396de91c-58a1-42d1-9385-04ea120b9b80 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:07.304470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:07.361695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:07.363044 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:07.363540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:07.363880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] PATH_INFO: `/v3/projects/1060fa71a3a747a4b9907433bb78fe1f` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:07.369058 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1060fa71a3a747a4b9907433bb78fe1f)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:07.371257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:07.430557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1060fa71a3a747a4b9907433bb78fe1f', 'request_id': 'req-cac3b2af-0851-4029-bae6-b9008d9f0af8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:07.439700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1060fa71a3a747a4b9907433bb78fe1f', 'request_id': 'req-cac3b2af-0851-4029-bae6-b9008d9f0af8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:07.454775 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1060fa71a3a747a4b9907433bb78fe1f', 'request_id': 'req-cac3b2af-0851-4029-bae6-b9008d9f0af8'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:07.469974 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] The token cache is being invalidate because project 1060fa71a3a747a4b9907433bb78fe1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:48:07.470266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:07.471223 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:07.472744 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1212/2409] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:48:07 2022] DELETE /identity/v3/projects/1060fa71a3a747a4b9907433bb78fe1f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:07.477364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:07.479112 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:07.496375 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dd10e-1c78-4f95-8e98-8d95371eb73e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:07.546390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:07.572008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:07.572876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:07.573221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:07.573701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] PATH_INFO: `/v3/users/16a43f2063254dc880d3ebba6d090c30` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:07.575124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a43f2063254dc880d3ebba6d090c30)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:07.577537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:07.631273 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a43f2063254dc880d3ebba6d090c30'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:07.639347 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a43f2063254dc880d3ebba6d090c30'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:07.640151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a43f2063254dc880d3ebba6d090c30'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:07.656101 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a43f2063254dc880d3ebba6d090c30'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:07.668318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-db4f7561-780a-4947-956c-f6d58347fcee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a43f2063254dc880d3ebba6d090c30'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:07.678701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1198/2410] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:48:07 2022] DELETE /identity/v3/users/16a43f2063254dc880d3ebba6d090c30 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:48:08.031306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:08.032980 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:08.050582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b2af-0851-4029-bae6-b9008d9f0af8 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:08.180553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:08.181481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:08.181864 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:08.182299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] PATH_INFO: `/v3/projects/76fdfe87be3e4d209731e1a4648daf87` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:08.184174 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] RBAC: Authorizing `identity:delete_project(project_id=76fdfe87be3e4d209731e1a4648daf87)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:08.186386 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:08.252287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76fdfe87be3e4d209731e1a4648daf87', 'request_id': 'req-34028c75-911f-4f2b-9549-6b6f6a92f47e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:08.254237 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:08.261838 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76fdfe87be3e4d209731e1a4648daf87', 'request_id': 'req-34028c75-911f-4f2b-9549-6b6f6a92f47e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:08.273060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76fdfe87be3e4d209731e1a4648daf87', 'request_id': 'req-34028c75-911f-4f2b-9549-6b6f6a92f47e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:08.286417 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] The token cache is being invalidate because project 76fdfe87be3e4d209731e1a4648daf87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:48:08.286810 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:08.287818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-34028c75-911f-4f2b-9549-6b6f6a92f47e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:08.289389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1213/2411] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:48:08 2022] DELETE /identity/v3/projects/76fdfe87be3e4d209731e1a4648daf87 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:08.294637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca6db1-7723-46a8-b900-e12524921699 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:08.295015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca6db1-7723-46a8-b900-e12524921699 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:08.295478 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca6db1-7723-46a8-b900-e12524921699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:08.326849 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-2eca6db1-7723-46a8-b900-e12524921699 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:08.328088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2eca6db1-7723-46a8-b900-e12524921699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:08.397918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1199/2412] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:08.406231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707c7a-554b-4452-b312-48f74860b1be None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:08.406735 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707c7a-554b-4452-b312-48f74860b1be None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:08.406867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707c7a-554b-4452-b312-48f74860b1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:08.431122 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-73707c7a-554b-4452-b312-48f74860b1be None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:08.432523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-73707c7a-554b-4452-b312-48f74860b1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:08.442342 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1214/2413] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:08.447159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca6db1-7723-46a8-b900-e12524921699 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:08.461298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca6db1-7723-46a8-b900-e12524921699 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:08.467525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69ba84-d4da-4bb8-baac-61c4c375488c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:08.468390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69ba84-d4da-4bb8-baac-61c4c375488c admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:08.469045 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69ba84-d4da-4bb8-baac-61c4c375488c admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:08.469414 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69ba84-d4da-4bb8-baac-61c4c375488c admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:08.470277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69ba84-d4da-4bb8-baac-61c4c375488c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:48:08.471106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69ba84-d4da-4bb8-baac-61c4c375488c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:08.473508 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69ba84-d4da-4bb8-baac-61c4c375488c admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:08.490650 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1200/2414] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:48:08.492703 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707c7a-554b-4452-b312-48f74860b1be None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:08.504162 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707c7a-554b-4452-b312-48f74860b1be None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:08.509729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30218-86be-4df2-82ee-fe2f612226f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:08.510518 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30218-86be-4df2-82ee-fe2f612226f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:08.510899 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30218-86be-4df2-82ee-fe2f612226f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:08.511311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30218-86be-4df2-82ee-fe2f612226f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:08.512057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e30218-86be-4df2-82ee-fe2f612226f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:08.536673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e30218-86be-4df2-82ee-fe2f612226f0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:08.581054 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1215/2415] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:48:08 2022] POST /identity/v3/projects => generated 371 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:48:08.585592 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69ba84-d4da-4bb8-baac-61c4c375488c admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:08.599503 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69ba84-d4da-4bb8-baac-61c4c375488c admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:08.662546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:08.685631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596bdb-c0a2-4305-9709-b79406e920bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:08.686371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596bdb-c0a2-4305-9709-b79406e920bf admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:08.686742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596bdb-c0a2-4305-9709-b79406e920bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:08.687189 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596bdb-c0a2-4305-9709-b79406e920bf admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:08.687911 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6596bdb-c0a2-4305-9709-b79406e920bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:08.690167 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6596bdb-c0a2-4305-9709-b79406e920bf admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:08.723005 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1201/2416] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:48:08 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:48:08.726895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30218-86be-4df2-82ee-fe2f612226f0 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:08.738120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30218-86be-4df2-82ee-fe2f612226f0 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:08.749057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93361eb3-62db-4f2e-9c92-f1d8c984745f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:08.749967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93361eb3-62db-4f2e-9c92-f1d8c984745f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:08.750406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93361eb3-62db-4f2e-9c92-f1d8c984745f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:08.750773 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93361eb3-62db-4f2e-9c92-f1d8c984745f admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:08.751865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93361eb3-62db-4f2e-9c92-f1d8c984745f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:08.755589 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93361eb3-62db-4f2e-9c92-f1d8c984745f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:08.771727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1216/2417] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:08.775809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596bdb-c0a2-4305-9709-b79406e920bf admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:08.786953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596bdb-c0a2-4305-9709-b79406e920bf admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:08.791941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce418a2-8645-4c17-98bd-3b33d3da833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:08.792540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce418a2-8645-4c17-98bd-3b33d3da833b admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:08.792817 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce418a2-8645-4c17-98bd-3b33d3da833b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:08.793139 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce418a2-8645-4c17-98bd-3b33d3da833b admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:08.793785 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce418a2-8645-4c17-98bd-3b33d3da833b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:08.795420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce418a2-8645-4c17-98bd-3b33d3da833b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:08.811652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1202/2418] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:08.816738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93361eb3-62db-4f2e-9c92-f1d8c984745f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:08.829110 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93361eb3-62db-4f2e-9c92-f1d8c984745f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:08.835450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc41a-9f31-4f90-8836-10a2a2ffd389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:08.836124 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc41a-9f31-4f90-8836-10a2a2ffd389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:08.836400 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc41a-9f31-4f90-8836-10a2a2ffd389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:08.836667 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fc41a-9f31-4f90-8836-10a2a2ffd389 admin admin] PATH_INFO: `/v3/projects/5fac208ee4e84c9195dd0b5c4fbbf43d/users/5fc7295bee4d48b99734d81d149d311c/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:08.857915 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fc41a-9f31-4f90-8836-10a2a2ffd389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fac208ee4e84c9195dd0b5c4fbbf43d, user_id=5fc7295bee4d48b99734d81d149d311c, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:08.860595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fc41a-9f31-4f90-8836-10a2a2ffd389 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:08.883372 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1217/2419] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:48:08 2022] PUT /identity/v3/projects/5fac208ee4e84c9195dd0b5c4fbbf43d/users/5fc7295bee4d48b99734d81d149d311c/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:48:08.895544 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343d47a-8560-4b34-87b5-d9cf9025cac1 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:08.896148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343d47a-8560-4b34-87b5-d9cf9025cac1 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:08.905506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343d47a-8560-4b34-87b5-d9cf9025cac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:08.937298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d343d47a-8560-4b34-87b5-d9cf9025cac1 None None] MFA Rules not processed for user `5fc7295bee4d48b99734d81d149d311c`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:08.944204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d343d47a-8560-4b34-87b5-d9cf9025cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:09.026268 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1203/2420] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:48:09.040077 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc41a-9f31-4f90-8836-10a2a2ffd389 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:09.042311 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7f2fc41a-9f31-4f90-8836-10a2a2ffd389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:09.128418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fc41a-9f31-4f90-8836-10a2a2ffd389 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:09.133256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00461ad4-08e5-41a5-ab20-91b1b4b930f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d2a655f4d6f45039d641133e3b56592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:09.133884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00461ad4-08e5-41a5-ab20-91b1b4b930f6 service glance] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:09.134164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00461ad4-08e5-41a5-ab20-91b1b4b930f6 service glance] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:09.134521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00461ad4-08e5-41a5-ab20-91b1b4b930f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:09.143756 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00461ad4-08e5-41a5-ab20-91b1b4b930f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:09.145251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00461ad4-08e5-41a5-ab20-91b1b4b930f6 service glance] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:09.159759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1218/2421] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:09.448162 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343d47a-8560-4b34-87b5-d9cf9025cac1 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:09.449757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d343d47a-8560-4b34-87b5-d9cf9025cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:09.512459 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343d47a-8560-4b34-87b5-d9cf9025cac1 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:09.516939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e097b-c0ad-421c-91c8-44608a194ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:09.517571 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e097b-c0ad-421c-91c8-44608a194ded service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:09.517834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e097b-c0ad-421c-91c8-44608a194ded service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:09.518109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e097b-c0ad-421c-91c8-44608a194ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:09.520803 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-285e097b-c0ad-421c-91c8-44608a194ded service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:09.608072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e097b-c0ad-421c-91c8-44608a194ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:09.609979 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e097b-c0ad-421c-91c8-44608a194ded service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:09.628328 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1204/2422] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:13.319375 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00461ad4-08e5-41a5-ab20-91b1b4b930f6 service glance] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:13.322567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-00461ad4-08e5-41a5-ab20-91b1b4b930f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:13.362689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00461ad4-08e5-41a5-ab20-91b1b4b930f6 service glance] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:13.368379 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:13.370121 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:13.370334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:13.370615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] PATH_INFO: `/v3/users/f76d70b021f44d1391653477c65f95af` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:13.372163 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76d70b021f44d1391653477c65f95af)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:13.378886 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:13.446702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76d70b021f44d1391653477c65f95af'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:13.474858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76d70b021f44d1391653477c65f95af'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:13.475747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76d70b021f44d1391653477c65f95af'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:13.487812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:13.490449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76d70b021f44d1391653477c65f95af'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:13.491513 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76d70b021f44d1391653477c65f95af'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:13.526865 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1219/2423] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:48:13 2022] DELETE /identity/v3/users/f76d70b021f44d1391653477c65f95af => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:13.753527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e097b-c0ad-421c-91c8-44608a194ded service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:13.760649 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:13.769377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e097b-c0ad-421c-91c8-44608a194ded service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:13.843721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:13.844666 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:13.845250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:13.845725 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] PATH_INFO: `/v3/projects/34d789b867f24711a6c1b5589a573560` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:13.847737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d789b867f24711a6c1b5589a573560)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:13.849981 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:13.910501 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d789b867f24711a6c1b5589a573560', 'request_id': 'req-6404664f-f17c-4a2c-a585-479d95314bd8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:13.918918 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d789b867f24711a6c1b5589a573560', 'request_id': 'req-6404664f-f17c-4a2c-a585-479d95314bd8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:13.928245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d789b867f24711a6c1b5589a573560', 'request_id': 'req-6404664f-f17c-4a2c-a585-479d95314bd8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:13.942088 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] The token cache is being invalidate because project 34d789b867f24711a6c1b5589a573560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:48:13.942436 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:13.943195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:13.948960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1205/2424] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:48:13 2022] DELETE /identity/v3/projects/34d789b867f24711a6c1b5589a573560 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:13.949875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:13.952918 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:13.968664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf0db-a729-4563-8924-baeff0629f23 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:14.009192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:14.010704 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:14.101132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:14.101132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:14.101132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:14.101132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] PATH_INFO: `/v3/users/a94981db1de14312ab19054745731580` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:14.101640 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a94981db1de14312ab19054745731580)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:14.101640 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:14.120724 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404664f-f17c-4a2c-a585-479d95314bd8 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:14.126011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:14.127418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:14.128810 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:14.129408 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] PATH_INFO: `/v3/users/72d748ebce2742058757304095d1a641` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:14.130786 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d748ebce2742058757304095d1a641)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:14.132824 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:14.166752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a94981db1de14312ab19054745731580'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.189415 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a94981db1de14312ab19054745731580'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.190941 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a94981db1de14312ab19054745731580'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.193878 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:14.201699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a94981db1de14312ab19054745731580'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.202635 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a94981db1de14312ab19054745731580'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.210065 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d748ebce2742058757304095d1a641'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.226399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1220/2425] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:48:13 2022] DELETE /identity/v3/users/a94981db1de14312ab19054745731580 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:48:14.226618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d748ebce2742058757304095d1a641'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.226618 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d748ebce2742058757304095d1a641'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.247022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d748ebce2742058757304095d1a641'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.258555 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d748ebce2742058757304095d1a641'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.265226 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1206/2426] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:48:14 2022] DELETE /identity/v3/users/72d748ebce2742058757304095d1a641 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:14.430593 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:14.432226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:14.466402 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:14.468332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:14.519270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23344453-e18f-4abe-90e1-96c3851836bf admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:14.534409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:14.553584 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fc59b-1357-4bdc-ab0a-d25d6221b849 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:14.558339 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:14.558977 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:14.559255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:14.559527 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] PATH_INFO: `/v3/projects/c84cfe68e5db4b2dba60443b48d23097` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:14.560638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84cfe68e5db4b2dba60443b48d23097)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:14.562577 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:14.582508 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:14.583139 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:14.583414 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:14.583681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] PATH_INFO: `/v3/projects/36417a0eb5b448f49f22434306fc0548` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:14.584774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36417a0eb5b448f49f22434306fc0548)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:14.587867 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:14.616184 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84cfe68e5db4b2dba60443b48d23097', 'request_id': 'req-b9f72e4b-36cc-4f54-8152-e8a656e399f0'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.632440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84cfe68e5db4b2dba60443b48d23097', 'request_id': 'req-b9f72e4b-36cc-4f54-8152-e8a656e399f0'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.640148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84cfe68e5db4b2dba60443b48d23097', 'request_id': 'req-b9f72e4b-36cc-4f54-8152-e8a656e399f0'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.643039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36417a0eb5b448f49f22434306fc0548', 'request_id': 'req-189c25d6-18c3-4db5-b171-813e3b27a867'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.648503 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36417a0eb5b448f49f22434306fc0548', 'request_id': 'req-189c25d6-18c3-4db5-b171-813e3b27a867'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.651652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] The token cache is being invalidate because project c84cfe68e5db4b2dba60443b48d23097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:48:14.651937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.652698 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-b9f72e4b-36cc-4f54-8152-e8a656e399f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.654433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1207/2427] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:48:14 2022] DELETE /identity/v3/projects/c84cfe68e5db4b2dba60443b48d23097 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:48:14.655585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36417a0eb5b448f49f22434306fc0548', 'request_id': 'req-189c25d6-18c3-4db5-b171-813e3b27a867'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.659382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2360fe-d625-43d2-9f9d-4429db4ce890 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:14.659662 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2360fe-d625-43d2-9f9d-4429db4ce890 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:14.659932 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2360fe-d625-43d2-9f9d-4429db4ce890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:14.669975 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] The token cache is being invalidate because project 36417a0eb5b448f49f22434306fc0548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:48:14.670352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.671075 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-189c25d6-18c3-4db5-b171-813e3b27a867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:14.672614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1221/2428] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:48:14 2022] DELETE /identity/v3/projects/36417a0eb5b448f49f22434306fc0548 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:14.679976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-6c2360fe-d625-43d2-9f9d-4429db4ce890 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:14.681278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-6c2360fe-d625-43d2-9f9d-4429db4ce890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:14.733907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:14.758530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1208/2429] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:48:14.766639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0888-ec2d-421b-897e-2c4eac81f953 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:14.766920 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0888-ec2d-421b-897e-2c4eac81f953 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:14.767197 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0888-ec2d-421b-897e-2c4eac81f953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:14.789235 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-af0b0888-ec2d-421b-897e-2c4eac81f953 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:14.790476 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-af0b0888-ec2d-421b-897e-2c4eac81f953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:14.799047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1222/2430] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:14.804018 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2360fe-d625-43d2-9f9d-4429db4ce890 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:14.814407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2360fe-d625-43d2-9f9d-4429db4ce890 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:14.819258 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85360a47-96d8-4963-a20a-6104304f6235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:14.820021 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85360a47-96d8-4963-a20a-6104304f6235 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:14.820334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85360a47-96d8-4963-a20a-6104304f6235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:14.820942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85360a47-96d8-4963-a20a-6104304f6235 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:14.821695 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85360a47-96d8-4963-a20a-6104304f6235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:48:14.822076 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85360a47-96d8-4963-a20a-6104304f6235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:14.824468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85360a47-96d8-4963-a20a-6104304f6235 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:14.847722 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1209/2431] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:48:14.848976 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b0888-ec2d-421b-897e-2c4eac81f953 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:14.858757 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b0888-ec2d-421b-897e-2c4eac81f953 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:14.864219 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd4b9-e79f-42e4-afa3-20a094c2ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:14.864906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd4b9-e79f-42e4-afa3-20a094c2ac9c admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:14.865192 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd4b9-e79f-42e4-afa3-20a094c2ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:14.865464 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd4b9-e79f-42e4-afa3-20a094c2ac9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:14.866108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fd4b9-e79f-42e4-afa3-20a094c2ac9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:14.868036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fd4b9-e79f-42e4-afa3-20a094c2ac9c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:14.900072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1223/2432] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:48:14 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:48:14.904374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85360a47-96d8-4963-a20a-6104304f6235 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:14.915815 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85360a47-96d8-4963-a20a-6104304f6235 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:14.985781 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c09ac-3de3-4cb4-8475-cfa006768dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:14.986312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c09ac-3de3-4cb4-8475-cfa006768dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:14.986590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c09ac-3de3-4cb4-8475-cfa006768dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:14.987522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c09ac-3de3-4cb4-8475-cfa006768dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:14.988153 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c09ac-3de3-4cb4-8475-cfa006768dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:14.990100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c09ac-3de3-4cb4-8475-cfa006768dc3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:15.010269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1210/2433] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:48:14 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:48:15.014596 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd4b9-e79f-42e4-afa3-20a094c2ac9c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:15.024862 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd4b9-e79f-42e4-afa3-20a094c2ac9c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:15.029539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78b72f-a85c-4062-a1bd-1e11a452bab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:15.030131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78b72f-a85c-4062-a1bd-1e11a452bab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:15.030408 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78b72f-a85c-4062-a1bd-1e11a452bab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:15.030691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78b72f-a85c-4062-a1bd-1e11a452bab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:15.031307 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78b72f-a85c-4062-a1bd-1e11a452bab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:15.032911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78b72f-a85c-4062-a1bd-1e11a452bab9 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:15.044633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1224/2434] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:15.049040 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c09ac-3de3-4cb4-8475-cfa006768dc3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:15.060278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c09ac-3de3-4cb4-8475-cfa006768dc3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:15.064949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8341a-87e7-4dfc-9e20-f092e4e744f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:15.065554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8341a-87e7-4dfc-9e20-f092e4e744f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:15.065832 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8341a-87e7-4dfc-9e20-f092e4e744f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:15.066108 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8341a-87e7-4dfc-9e20-f092e4e744f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:15.066744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8341a-87e7-4dfc-9e20-f092e4e744f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:15.068507 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8341a-87e7-4dfc-9e20-f092e4e744f7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:15.079313 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1211/2435] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:15.083653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78b72f-a85c-4062-a1bd-1e11a452bab9 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:15.093854 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78b72f-a85c-4062-a1bd-1e11a452bab9 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:15.099102 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e24d18-0099-4d76-8ba5-c2e776fb7ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:15.099751 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e24d18-0099-4d76-8ba5-c2e776fb7ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:15.100031 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e24d18-0099-4d76-8ba5-c2e776fb7ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:15.100305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e24d18-0099-4d76-8ba5-c2e776fb7ea1 admin admin] PATH_INFO: `/v3/projects/16d58c86b53f4addbecea9ced3d26716/users/400fe30a3db543ed80fe07a0a1089cc6/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:15.116203 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e24d18-0099-4d76-8ba5-c2e776fb7ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d58c86b53f4addbecea9ced3d26716, user_id=400fe30a3db543ed80fe07a0a1089cc6, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:15.118923 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e24d18-0099-4d76-8ba5-c2e776fb7ea1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:15.132572 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1225/2436] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:48:15 2022] PUT /identity/v3/projects/16d58c86b53f4addbecea9ced3d26716/users/400fe30a3db543ed80fe07a0a1089cc6/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:48:15.142802 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ed66d-8ed8-4e21-8308-5a6e6a5cb56a None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:15.143085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ed66d-8ed8-4e21-8308-5a6e6a5cb56a None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:15.143354 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ed66d-8ed8-4e21-8308-5a6e6a5cb56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:15.173820 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-df2ed66d-8ed8-4e21-8308-5a6e6a5cb56a None None] MFA Rules not processed for user `400fe30a3db543ed80fe07a0a1089cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:15.175708 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-df2ed66d-8ed8-4e21-8308-5a6e6a5cb56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:15.245818 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1212/2437] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:15.257042 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e24d18-0099-4d76-8ba5-c2e776fb7ea1 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:15.258773 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-81e24d18-0099-4d76-8ba5-c2e776fb7ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:15.311305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:15.345038 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e24d18-0099-4d76-8ba5-c2e776fb7ea1 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:15.351707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc781a02-9a97-4408-bfe1-eb2054426be2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:15.357126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc781a02-9a97-4408-bfe1-eb2054426be2 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:15.357126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc781a02-9a97-4408-bfe1-eb2054426be2 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:15.357126 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc781a02-9a97-4408-bfe1-eb2054426be2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:15.367623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc781a02-9a97-4408-bfe1-eb2054426be2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:15.370200 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc781a02-9a97-4408-bfe1-eb2054426be2 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:15.381800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1226/2438] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:18.103490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ed66d-8ed8-4e21-8308-5a6e6a5cb56a None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:18.145828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:18.167616 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ed66d-8ed8-4e21-8308-5a6e6a5cb56a None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:18.172185 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:18.172811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:18.173113 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:18.173380 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] PATH_INFO: `/v3/users/400fe30a3db543ed80fe07a0a1089cc6` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:18.175117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=400fe30a3db543ed80fe07a0a1089cc6)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:18.177113 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:18.226157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400fe30a3db543ed80fe07a0a1089cc6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:18.233343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400fe30a3db543ed80fe07a0a1089cc6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:18.234012 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400fe30a3db543ed80fe07a0a1089cc6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:18.247869 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400fe30a3db543ed80fe07a0a1089cc6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:18.254491 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400fe30a3db543ed80fe07a0a1089cc6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:18.260596 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1213/2439] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:48:18 2022] DELETE /identity/v3/users/400fe30a3db543ed80fe07a0a1089cc6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:18.265242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41d3b-e974-4831-aa8d-c3ceca9cf943 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:18.265747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41d3b-e974-4831-aa8d-c3ceca9cf943 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:18.265850 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41d3b-e974-4831-aa8d-c3ceca9cf943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:18.286287 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-edc41d3b-e974-4831-aa8d-c3ceca9cf943 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:18.287471 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-edc41d3b-e974-4831-aa8d-c3ceca9cf943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:18.348797 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1227/2440] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:18.358450 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:18.359897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:18.416787 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a543-9f04-437f-90b3-d01759550f74 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:18.420377 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc4676-c692-43cf-95e1-97960b7c93ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:18.421006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc4676-c692-43cf-95e1-97960b7c93ce service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:18.421286 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc4676-c692-43cf-95e1-97960b7c93ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:18.421558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc4676-c692-43cf-95e1-97960b7c93ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:18.430364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc4676-c692-43cf-95e1-97960b7c93ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:18.431807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc4676-c692-43cf-95e1-97960b7c93ce service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:18.441984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1214/2441] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:18.712919 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc41d3b-e974-4831-aa8d-c3ceca9cf943 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:18.714336 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-edc41d3b-e974-4831-aa8d-c3ceca9cf943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:18.726114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc41d3b-e974-4831-aa8d-c3ceca9cf943 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:18.730785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:18.732093 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:18.732386 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:18.732929 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] PATH_INFO: `/v3/projects/16d58c86b53f4addbecea9ced3d26716` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:18.734062 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16d58c86b53f4addbecea9ced3d26716)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:18.736131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:18.787858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16d58c86b53f4addbecea9ced3d26716', 'request_id': 'req-9b8c7bbf-6385-4fc5-8bf9-889946b09934'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:18.793331 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16d58c86b53f4addbecea9ced3d26716', 'request_id': 'req-9b8c7bbf-6385-4fc5-8bf9-889946b09934'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:18.800814 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16d58c86b53f4addbecea9ced3d26716', 'request_id': 'req-9b8c7bbf-6385-4fc5-8bf9-889946b09934'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:18.812083 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] The token cache is being invalidate because project 16d58c86b53f4addbecea9ced3d26716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:48:18.812368 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:18.813031 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9b8c7bbf-6385-4fc5-8bf9-889946b09934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:18.814502 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1228/2442] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:48:18 2022] DELETE /identity/v3/projects/16d58c86b53f4addbecea9ced3d26716 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:18.819826 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926cb271-f77f-4e37-bb58-1cde1b38e294 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:18.820138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926cb271-f77f-4e37-bb58-1cde1b38e294 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:18.820410 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926cb271-f77f-4e37-bb58-1cde1b38e294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:18.840541 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-926cb271-f77f-4e37-bb58-1cde1b38e294 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:18.841782 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-926cb271-f77f-4e37-bb58-1cde1b38e294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:18.902006 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1215/2443] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:18.909610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db15e1f-2a9b-490a-bdf6-49d5e2d34674 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:18.909901 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db15e1f-2a9b-490a-bdf6-49d5e2d34674 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:18.910180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db15e1f-2a9b-490a-bdf6-49d5e2d34674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:18.930222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-9db15e1f-2a9b-490a-bdf6-49d5e2d34674 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:18.931424 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9db15e1f-2a9b-490a-bdf6-49d5e2d34674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:18.939270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1229/2444] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:18.943892 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926cb271-f77f-4e37-bb58-1cde1b38e294 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:18.948912 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:18.955531 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926cb271-f77f-4e37-bb58-1cde1b38e294 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:18.961109 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7ca3e-d4d2-4b93-a1a8-3dddf1398c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:18.961736 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7ca3e-d4d2-4b93-a1a8-3dddf1398c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:18.962014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7ca3e-d4d2-4b93-a1a8-3dddf1398c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:18.962317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7ca3e-d4d2-4b93-a1a8-3dddf1398c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:18.962956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7ca3e-d4d2-4b93-a1a8-3dddf1398c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:48:18.963284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7ca3e-d4d2-4b93-a1a8-3dddf1398c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:18.964947 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7ca3e-d4d2-4b93-a1a8-3dddf1398c96 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:18.976579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1216/2445] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:48:18.981013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db15e1f-2a9b-490a-bdf6-49d5e2d34674 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:18.992080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db15e1f-2a9b-490a-bdf6-49d5e2d34674 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:18.996922 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c08d94-c3fa-43e9-a5e3-6af848088531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:18.997528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c08d94-c3fa-43e9-a5e3-6af848088531 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:18.997802 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c08d94-c3fa-43e9-a5e3-6af848088531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:18.998067 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c08d94-c3fa-43e9-a5e3-6af848088531 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:18.998699 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c08d94-c3fa-43e9-a5e3-6af848088531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.000577 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c08d94-c3fa-43e9-a5e3-6af848088531 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.028260 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1230/2446] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:48:18 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:48:19.032502 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7ca3e-d4d2-4b93-a1a8-3dddf1398c96 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:19.042727 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7ca3e-d4d2-4b93-a1a8-3dddf1398c96 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:19.109426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d4a7d-3b5d-4d5b-9c5b-b57673885817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:19.110034 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d4a7d-3b5d-4d5b-9c5b-b57673885817 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.110305 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d4a7d-3b5d-4d5b-9c5b-b57673885817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.110570 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d4a7d-3b5d-4d5b-9c5b-b57673885817 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.111204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d4a7d-3b5d-4d5b-9c5b-b57673885817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.114259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d4a7d-3b5d-4d5b-9c5b-b57673885817 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.135945 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1217/2447] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:48:19 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:48:19.140073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c08d94-c3fa-43e9-a5e3-6af848088531 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:19.150675 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c08d94-c3fa-43e9-a5e3-6af848088531 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:19.155815 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15714411-99b6-430f-8cc3-b3b54f1ffe1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:19.156523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15714411-99b6-430f-8cc3-b3b54f1ffe1f admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.156908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15714411-99b6-430f-8cc3-b3b54f1ffe1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.157193 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15714411-99b6-430f-8cc3-b3b54f1ffe1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.157850 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15714411-99b6-430f-8cc3-b3b54f1ffe1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.159469 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15714411-99b6-430f-8cc3-b3b54f1ffe1f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.174552 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1231/2448] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:48:19.179431 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d4a7d-3b5d-4d5b-9c5b-b57673885817 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:19.189565 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d4a7d-3b5d-4d5b-9c5b-b57673885817 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:19.196632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37cd6f-1fa0-48cd-a945-d0bfbb12144f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:19.197741 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37cd6f-1fa0-48cd-a945-d0bfbb12144f admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.198050 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37cd6f-1fa0-48cd-a945-d0bfbb12144f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.198283 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37cd6f-1fa0-48cd-a945-d0bfbb12144f admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.198938 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37cd6f-1fa0-48cd-a945-d0bfbb12144f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.200807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37cd6f-1fa0-48cd-a945-d0bfbb12144f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.216176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1218/2449] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:19.220393 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15714411-99b6-430f-8cc3-b3b54f1ffe1f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:19.230433 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15714411-99b6-430f-8cc3-b3b54f1ffe1f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:19.235292 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb9dda-763c-4e2d-a5e2-6e1d637756bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:19.235943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb9dda-763c-4e2d-a5e2-6e1d637756bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.236226 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb9dda-763c-4e2d-a5e2-6e1d637756bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.236504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb9dda-763c-4e2d-a5e2-6e1d637756bd admin admin] PATH_INFO: `/v3/projects/6e4630d42ccd4fd8b4f8e38b234e58f6/users/0aa68926bfc443958b8b5e954883237b/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.246794 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:19.255131 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb9dda-763c-4e2d-a5e2-6e1d637756bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4630d42ccd4fd8b4f8e38b234e58f6, user_id=0aa68926bfc443958b8b5e954883237b, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.258292 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb9dda-763c-4e2d-a5e2-6e1d637756bd admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.272206 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1232/2450] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:48:19 2022] PUT /identity/v3/projects/6e4630d42ccd4fd8b4f8e38b234e58f6/users/0aa68926bfc443958b8b5e954883237b/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:19.282601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57882b-78f3-46d5-a9de-196844ee09ac None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.282898 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57882b-78f3-46d5-a9de-196844ee09ac None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.283173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57882b-78f3-46d5-a9de-196844ee09ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.314740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-ba57882b-78f3-46d5-a9de-196844ee09ac None None] MFA Rules not processed for user `0aa68926bfc443958b8b5e954883237b`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:19.316141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-ba57882b-78f3-46d5-a9de-196844ee09ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:19.390713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1219/2451] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:19.395189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb9dda-763c-4e2d-a5e2-6e1d637756bd admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:19.472332 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb9dda-763c-4e2d-a5e2-6e1d637756bd admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:19.478169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a9ca3-da66-42d4-b1f0-6b99fe32ee20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:19.478754 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a9ca3-da66-42d4-b1f0-6b99fe32ee20 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.479032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a9ca3-da66-42d4-b1f0-6b99fe32ee20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.479301 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a9ca3-da66-42d4-b1f0-6b99fe32ee20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.479924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a9ca3-da66-42d4-b1f0-6b99fe32ee20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.481836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a9ca3-da66-42d4-b1f0-6b99fe32ee20 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.517847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1233/2452] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:48:19 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:48:19.521632 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57882b-78f3-46d5-a9de-196844ee09ac None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:19.536342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57882b-78f3-46d5-a9de-196844ee09ac None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:19.609633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:19.617879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0ca47-f8d5-4dc2-b441-43e0d96907f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:19.618530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0ca47-f8d5-4dc2-b441-43e0d96907f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.618992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0ca47-f8d5-4dc2-b441-43e0d96907f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.619448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0ca47-f8d5-4dc2-b441-43e0d96907f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.620090 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0ca47-f8d5-4dc2-b441-43e0d96907f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.622111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0ca47-f8d5-4dc2-b441-43e0d96907f3 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.664684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1220/2453] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:48:19 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:48:19.668580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a9ca3-da66-42d4-b1f0-6b99fe32ee20 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:19.678896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a9ca3-da66-42d4-b1f0-6b99fe32ee20 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:19.683660 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76cfb8-a7c2-4db5-90d6-e7a9e4fcbcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:19.684304 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76cfb8-a7c2-4db5-90d6-e7a9e4fcbcac admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.684591 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76cfb8-a7c2-4db5-90d6-e7a9e4fcbcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.684887 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76cfb8-a7c2-4db5-90d6-e7a9e4fcbcac admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.685565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76cfb8-a7c2-4db5-90d6-e7a9e4fcbcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.687153 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76cfb8-a7c2-4db5-90d6-e7a9e4fcbcac admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.700752 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1234/2454] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:19.702631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0ca47-f8d5-4dc2-b441-43e0d96907f3 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:19.713599 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0ca47-f8d5-4dc2-b441-43e0d96907f3 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:19.718356 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62df1d1-0dcf-42b0-b556-f8c057806fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:19.719214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62df1d1-0dcf-42b0-b556-f8c057806fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.719924 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62df1d1-0dcf-42b0-b556-f8c057806fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.720202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62df1d1-0dcf-42b0-b556-f8c057806fe4 admin admin] PATH_INFO: `/v3/projects/38c187f4805b48329a017d8e43cc302b/users/a450bb86d50a49638421147e2f2d9276/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.744004 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62df1d1-0dcf-42b0-b556-f8c057806fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c187f4805b48329a017d8e43cc302b, user_id=a450bb86d50a49638421147e2f2d9276, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.746879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62df1d1-0dcf-42b0-b556-f8c057806fe4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.759556 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1221/2455] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:48:19 2022] PUT /identity/v3/projects/38c187f4805b48329a017d8e43cc302b/users/a450bb86d50a49638421147e2f2d9276/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:19.763592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76cfb8-a7c2-4db5-90d6-e7a9e4fcbcac admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:19.816749 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76cfb8-a7c2-4db5-90d6-e7a9e4fcbcac admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:19.877472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba36fa-0eab-49cc-9e3b-f3631286285b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:19.878113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba36fa-0eab-49cc-9e3b-f3631286285b admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.878392 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba36fa-0eab-49cc-9e3b-f3631286285b admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.878663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba36fa-0eab-49cc-9e3b-f3631286285b admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.879303 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba36fa-0eab-49cc-9e3b-f3631286285b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.880925 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba36fa-0eab-49cc-9e3b-f3631286285b admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.890753 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1235/2456] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:19.895546 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62df1d1-0dcf-42b0-b556-f8c057806fe4 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:19.933156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62df1d1-0dcf-42b0-b556-f8c057806fe4 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:19.938089 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bf478-7f49-40e1-9e31-0c7374a28a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:19.938761 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bf478-7f49-40e1-9e31-0c7374a28a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.939036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bf478-7f49-40e1-9e31-0c7374a28a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.939308 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bf478-7f49-40e1-9e31-0c7374a28a37 admin admin] PATH_INFO: `/v3/projects/38c187f4805b48329a017d8e43cc302b/users/a450bb86d50a49638421147e2f2d9276/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.941272 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bf478-7f49-40e1-9e31-0c7374a28a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c187f4805b48329a017d8e43cc302b, user_id=a450bb86d50a49638421147e2f2d9276, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:19.943937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bf478-7f49-40e1-9e31-0c7374a28a37 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:19.957066 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1222/2457] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:48:19 2022] PUT /identity/v3/projects/38c187f4805b48329a017d8e43cc302b/users/a450bb86d50a49638421147e2f2d9276/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:19.968652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235e8c8-24cf-4831-b1da-7357153e8204 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:19.969133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235e8c8-24cf-4831-b1da-7357153e8204 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:19.969352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235e8c8-24cf-4831-b1da-7357153e8204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:19.999171 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-9235e8c8-24cf-4831-b1da-7357153e8204 None None] MFA Rules not processed for user `a450bb86d50a49638421147e2f2d9276`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:20.000462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9235e8c8-24cf-4831-b1da-7357153e8204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:20.006057 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:20.071279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1236/2458] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:20.083771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bf478-7f49-40e1-9e31-0c7374a28a37 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:20.085525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2d6bf478-7f49-40e1-9e31-0c7374a28a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:20.159877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bf478-7f49-40e1-9e31-0c7374a28a37 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:20.163765 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f12e1-d44e-4bfa-aa6d-a469e202d27e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:20.164379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f12e1-d44e-4bfa-aa6d-a469e202d27e service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:20.164877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f12e1-d44e-4bfa-aa6d-a469e202d27e service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:20.165230 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f12e1-d44e-4bfa-aa6d-a469e202d27e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:20.176513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f12e1-d44e-4bfa-aa6d-a469e202d27e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:20.178002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f12e1-d44e-4bfa-aa6d-a469e202d27e service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:20.187974 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1223/2459] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:48:20.270879 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235e8c8-24cf-4831-b1da-7357153e8204 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:20.280651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235e8c8-24cf-4831-b1da-7357153e8204 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:20.284243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019fdbc-b4ec-4f88-ae70-796fad04bc5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:20.284859 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019fdbc-b4ec-4f88-ae70-796fad04bc5a service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:20.285151 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019fdbc-b4ec-4f88-ae70-796fad04bc5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:20.285425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019fdbc-b4ec-4f88-ae70-796fad04bc5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:20.344530 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019fdbc-b4ec-4f88-ae70-796fad04bc5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:20.346055 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019fdbc-b4ec-4f88-ae70-796fad04bc5a service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:20.354927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1237/2460] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:30.183582 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:30.350843 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=54 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:32.742720 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f12e1-d44e-4bfa-aa6d-a469e202d27e service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:32.744180 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-c09f12e1-d44e-4bfa-aa6d-a469e202d27e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:32.834255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f12e1-d44e-4bfa-aa6d-a469e202d27e service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:32.839360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629be499-b287-488f-b1de-8dd5a78e1b7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:32.839969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be499-b287-488f-b1de-8dd5a78e1b7e service placement] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:32.840243 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be499-b287-488f-b1de-8dd5a78e1b7e service placement] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:32.840522 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be499-b287-488f-b1de-8dd5a78e1b7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:32.842316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-629be499-b287-488f-b1de-8dd5a78e1b7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:32.855813 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629be499-b287-488f-b1de-8dd5a78e1b7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:32.857678 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629be499-b287-488f-b1de-8dd5a78e1b7e service placement] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:32.868875 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1224/2461] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:42.537515 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019fdbc-b4ec-4f88-ae70-796fad04bc5a service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:42.538935 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d019fdbc-b4ec-4f88-ae70-796fad04bc5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:42.611661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019fdbc-b4ec-4f88-ae70-796fad04bc5a service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:42.617232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:42.617935 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:42.618207 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:42.618478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] PATH_INFO: `/v3/users/5fc7295bee4d48b99734d81d149d311c` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:42.619610 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fc7295bee4d48b99734d81d149d311c)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:42.621641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:42.669673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fc7295bee4d48b99734d81d149d311c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:42.681967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fc7295bee4d48b99734d81d149d311c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:42.682634 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fc7295bee4d48b99734d81d149d311c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:42.689306 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fc7295bee4d48b99734d81d149d311c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:42.689924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fc7295bee4d48b99734d81d149d311c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:42.703177 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1238/2462] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:48:42 2022] DELETE /identity/v3/users/5fc7295bee4d48b99734d81d149d311c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:42.707792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca255bc-750d-4170-9e8e-edffd26e4c6f None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:42.708295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca255bc-750d-4170-9e8e-edffd26e4c6f None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:42.708403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca255bc-750d-4170-9e8e-edffd26e4c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:42.735737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-fca255bc-750d-4170-9e8e-edffd26e4c6f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:42.737493 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fca255bc-750d-4170-9e8e-edffd26e4c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:42.794519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1225/2463] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:42.804683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:42.806132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:42.885008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4b3a8-5b4b-4f9d-9d79-86e5515abded admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:42.888655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e7ae8-c4e9-41fa-86e5-dc298bbbf1af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:42.889279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e7ae8-c4e9-41fa-86e5-dc298bbbf1af service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:42.889560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e7ae8-c4e9-41fa-86e5-dc298bbbf1af service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:42.889836 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e7ae8-c4e9-41fa-86e5-dc298bbbf1af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:42.898730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e7ae8-c4e9-41fa-86e5-dc298bbbf1af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:42.900146 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e7ae8-c4e9-41fa-86e5-dc298bbbf1af service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:42.909930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1239/2464] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:43.173808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca255bc-750d-4170-9e8e-edffd26e4c6f None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:43.175870 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-fca255bc-750d-4170-9e8e-edffd26e4c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:43.188349 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca255bc-750d-4170-9e8e-edffd26e4c6f None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:43.192934 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:43.193564 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:43.193851 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:43.194105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] PATH_INFO: `/v3/projects/5fac208ee4e84c9195dd0b5c4fbbf43d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:43.195218 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fac208ee4e84c9195dd0b5c4fbbf43d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:43.197157 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:43.257927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fac208ee4e84c9195dd0b5c4fbbf43d', 'request_id': 'req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:43.257927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fac208ee4e84c9195dd0b5c4fbbf43d', 'request_id': 'req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:43.257927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:43.260016 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fac208ee4e84c9195dd0b5c4fbbf43d', 'request_id': 'req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:43.273259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] The token cache is being invalidate because project 5fac208ee4e84c9195dd0b5c4fbbf43d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:48:43.273549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:43.274202 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8cabf78e-f10d-4cf2-bf2d-6447e9b400a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:43.275668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1226/2465] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:48:43 2022] DELETE /identity/v3/projects/5fac208ee4e84c9195dd0b5c4fbbf43d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:43.280465 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee9831-8758-4e59-b0fa-df9136be48b9 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:43.280750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee9831-8758-4e59-b0fa-df9136be48b9 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:43.281065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee9831-8758-4e59-b0fa-df9136be48b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:43.286617 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:43.337358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-ecee9831-8758-4e59-b0fa-df9136be48b9 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:43.338558 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-ecee9831-8758-4e59-b0fa-df9136be48b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:43.401904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1240/2466] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:43.464196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02ef7e-c298-4478-a1a6-6ac7f71c8ce2 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:43.464480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02ef7e-c298-4478-a1a6-6ac7f71c8ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:43.464755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02ef7e-c298-4478-a1a6-6ac7f71c8ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:43.484719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-1e02ef7e-c298-4478-a1a6-6ac7f71c8ce2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:43.485937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-1e02ef7e-c298-4478-a1a6-6ac7f71c8ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:43.494024 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1227/2467] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:43.498256 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee9831-8758-4e59-b0fa-df9136be48b9 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:43.507951 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee9831-8758-4e59-b0fa-df9136be48b9 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:43.512944 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff022f27-655a-48a3-a69a-8dd3e8598a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:43.513546 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff022f27-655a-48a3-a69a-8dd3e8598a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:43.513871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff022f27-655a-48a3-a69a-8dd3e8598a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:43.514141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff022f27-655a-48a3-a69a-8dd3e8598a7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:43.514783 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff022f27-655a-48a3-a69a-8dd3e8598a7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:48:43.515136 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff022f27-655a-48a3-a69a-8dd3e8598a7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:43.516807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff022f27-655a-48a3-a69a-8dd3e8598a7c admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:43.528995 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1241/2468] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:48:43.534297 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02ef7e-c298-4478-a1a6-6ac7f71c8ce2 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:43.543860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02ef7e-c298-4478-a1a6-6ac7f71c8ce2 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:43.548955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b144807-49ab-4ab5-b17e-edc36fb7181c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:43.549641 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b144807-49ab-4ab5-b17e-edc36fb7181c admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:43.549969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b144807-49ab-4ab5-b17e-edc36fb7181c admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:43.550188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b144807-49ab-4ab5-b17e-edc36fb7181c admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:43.550823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b144807-49ab-4ab5-b17e-edc36fb7181c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:43.552771 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b144807-49ab-4ab5-b17e-edc36fb7181c admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:43.584220 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1228/2469] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:48:43 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:48:43.588695 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff022f27-655a-48a3-a69a-8dd3e8598a7c admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:43.599327 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff022f27-655a-48a3-a69a-8dd3e8598a7c admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:43.655657 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee025f-3695-4ef6-ba71-8636f88c2809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:43.656271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee025f-3695-4ef6-ba71-8636f88c2809 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:43.656545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee025f-3695-4ef6-ba71-8636f88c2809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:43.656813 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee025f-3695-4ef6-ba71-8636f88c2809 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:43.657475 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee025f-3695-4ef6-ba71-8636f88c2809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:43.659969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee025f-3695-4ef6-ba71-8636f88c2809 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:43.681609 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1242/2470] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:48:43 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:48:43.685757 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b144807-49ab-4ab5-b17e-edc36fb7181c admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:43.696492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b144807-49ab-4ab5-b17e-edc36fb7181c admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:43.701815 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de082d20-bc85-48b3-872f-75d59d25715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:43.702422 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de082d20-bc85-48b3-872f-75d59d25715b admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:43.702704 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de082d20-bc85-48b3-872f-75d59d25715b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:43.702976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de082d20-bc85-48b3-872f-75d59d25715b admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:43.703620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de082d20-bc85-48b3-872f-75d59d25715b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:43.705358 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de082d20-bc85-48b3-872f-75d59d25715b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:43.716161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1229/2471] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:43.720208 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee025f-3695-4ef6-ba71-8636f88c2809 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:43.730346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee025f-3695-4ef6-ba71-8636f88c2809 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:43.735362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5468a3-162a-4d2b-9289-da9cb64441d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:43.735832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5468a3-162a-4d2b-9289-da9cb64441d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:43.736114 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5468a3-162a-4d2b-9289-da9cb64441d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:43.736938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5468a3-162a-4d2b-9289-da9cb64441d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:43.737170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5468a3-162a-4d2b-9289-da9cb64441d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:43.738645 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5468a3-162a-4d2b-9289-da9cb64441d0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:43.749466 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1243/2472] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:48:43.754160 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de082d20-bc85-48b3-872f-75d59d25715b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:43.764464 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de082d20-bc85-48b3-872f-75d59d25715b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:43.770271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d98cb-1d43-470e-9520-cca285739f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:43.771056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d98cb-1d43-470e-9520-cca285739f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:43.771334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d98cb-1d43-470e-9520-cca285739f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:43.771648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d98cb-1d43-470e-9520-cca285739f5f admin admin] PATH_INFO: `/v3/projects/2bd0a559530442a6aebd24d57babf444/users/e808446bce0e474a93d31c9e8ee473b4/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:43.789077 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d98cb-1d43-470e-9520-cca285739f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd0a559530442a6aebd24d57babf444, user_id=e808446bce0e474a93d31c9e8ee473b4, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:43.789193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d98cb-1d43-470e-9520-cca285739f5f admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:43.802594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1230/2473] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:48:43 2022] PUT /identity/v3/projects/2bd0a559530442a6aebd24d57babf444/users/e808446bce0e474a93d31c9e8ee473b4/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:48:43.812381 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81a459-1da2-4a4f-a2a8-d0951954e84f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:43.812674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81a459-1da2-4a4f-a2a8-d0951954e84f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:43.812984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81a459-1da2-4a4f-a2a8-d0951954e84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:43.843668 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-2b81a459-1da2-4a4f-a2a8-d0951954e84f None None] MFA Rules not processed for user `e808446bce0e474a93d31c9e8ee473b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:43.845034 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2b81a459-1da2-4a4f-a2a8-d0951954e84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:43.879039 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:43.917080 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1244/2474] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:43.928897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d98cb-1d43-470e-9520-cca285739f5f admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:43.930753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-456d98cb-1d43-470e-9520-cca285739f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:43.993495 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d98cb-1d43-470e-9520-cca285739f5f admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:44.000597 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823bd8c-bbd9-4635-830c-9c9da6aa9c98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:44.001302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823bd8c-bbd9-4635-830c-9c9da6aa9c98 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:44.001710 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823bd8c-bbd9-4635-830c-9c9da6aa9c98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:44.001844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823bd8c-bbd9-4635-830c-9c9da6aa9c98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:44.011194 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823bd8c-bbd9-4635-830c-9c9da6aa9c98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:44.012627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823bd8c-bbd9-4635-830c-9c9da6aa9c98 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:44.022036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1231/2475] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:51.992575 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81a459-1da2-4a4f-a2a8-d0951954e84f None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:51.994041 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-2b81a459-1da2-4a4f-a2a8-d0951954e84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:52.057384 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81a459-1da2-4a4f-a2a8-d0951954e84f None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:52.061841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:52.062462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:52.062738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:52.063005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] PATH_INFO: `/v3/users/6221c883d64b43349bf01523e5921384` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:52.064106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6221c883d64b43349bf01523e5921384)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:52.066196 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:52.109908 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6221c883d64b43349bf01523e5921384'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:52.121830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6221c883d64b43349bf01523e5921384'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:52.122479 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6221c883d64b43349bf01523e5921384'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:52.129205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6221c883d64b43349bf01523e5921384'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:52.129830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6221c883d64b43349bf01523e5921384'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:52.141691 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1245/2476] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:48:51 2022] DELETE /identity/v3/users/6221c883d64b43349bf01523e5921384 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:48:52.146152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0528d-469a-472e-aa4e-d7bd05ff9950 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:52.146606 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0528d-469a-472e-aa4e-d7bd05ff9950 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:52.146705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0528d-469a-472e-aa4e-d7bd05ff9950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:52.165390 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9da0528d-469a-472e-aa4e-d7bd05ff9950 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:48:52.166572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9da0528d-469a-472e-aa4e-d7bd05ff9950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:52.225144 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1232/2477] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:48:52.234702 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:52.236089 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:52.293108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d885dc7-1cb5-4cd1-9ff9-c04a65b3e9ae admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:52.296712 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d017451-93e2-4022-a15d-083c5d7e3a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:52.297344 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d017451-93e2-4022-a15d-083c5d7e3a02 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:52.297637 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d017451-93e2-4022-a15d-083c5d7e3a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:52.297897 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d017451-93e2-4022-a15d-083c5d7e3a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:52.304662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:52.306967 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d017451-93e2-4022-a15d-083c5d7e3a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:52.308884 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d017451-93e2-4022-a15d-083c5d7e3a02 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:52.319866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1246/2478] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:48:52.613657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0528d-469a-472e-aa4e-d7bd05ff9950 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:48:52.615685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9da0528d-469a-472e-aa4e-d7bd05ff9950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:48:52.633984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0528d-469a-472e-aa4e-d7bd05ff9950 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:48:52.641556 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:48:52.642196 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:48:52.642472 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:48:52.643446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] PATH_INFO: `/v3/projects/670892b833e0414eb6d4b639d85f98c6` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:48:52.644823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=670892b833e0414eb6d4b639d85f98c6)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:48:52.647036 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:48:52.678318 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:48:52.715017 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670892b833e0414eb6d4b639d85f98c6', 'request_id': 'req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:52.721583 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670892b833e0414eb6d4b639d85f98c6', 'request_id': 'req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:52.730055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670892b833e0414eb6d4b639d85f98c6', 'request_id': 'req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:52.743838 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] The token cache is being invalidate because project 670892b833e0414eb6d4b639d85f98c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:48:52.744056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:52.744709 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-ef2c2f3c-6f59-4c1d-8138-6ac4a4251950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:48:52.746513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1233/2479] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:48:52 2022] DELETE /identity/v3/projects/670892b833e0414eb6d4b639d85f98c6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:00.241739 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d017451-93e2-4022-a15d-083c5d7e3a02 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:00.243149 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0d017451-93e2-4022-a15d-083c5d7e3a02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:00.255746 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d017451-93e2-4022-a15d-083c5d7e3a02 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:00.316674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:00.317338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:00.317619 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:00.317893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] PATH_INFO: `/v3/users/e808446bce0e474a93d31c9e8ee473b4` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:00.319022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e808446bce0e474a93d31c9e8ee473b4)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:00.320949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:00.366291 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e808446bce0e474a93d31c9e8ee473b4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:00.379457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e808446bce0e474a93d31c9e8ee473b4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:00.380236 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e808446bce0e474a93d31c9e8ee473b4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:00.387978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e808446bce0e474a93d31c9e8ee473b4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:00.388658 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e808446bce0e474a93d31c9e8ee473b4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:00.400779 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1247/2480] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:49:00 2022] DELETE /identity/v3/users/e808446bce0e474a93d31c9e8ee473b4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:00.405302 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8080f-88c4-42d6-b835-78d4f7c0e96d None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:00.405807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8080f-88c4-42d6-b835-78d4f7c0e96d None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:00.405929 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8080f-88c4-42d6-b835-78d4f7c0e96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:00.425631 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-26f8080f-88c4-42d6-b835-78d4f7c0e96d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:00.426823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-26f8080f-88c4-42d6-b835-78d4f7c0e96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:00.483790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1234/2481] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:00.494189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:00.495573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:00.552036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e445a-9985-4e0f-818d-44197a580c52 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:00.555737 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd39a2c-03f8-4a7d-942b-48f916d18849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:00.556339 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd39a2c-03f8-4a7d-942b-48f916d18849 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:00.556618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd39a2c-03f8-4a7d-942b-48f916d18849 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:00.556916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd39a2c-03f8-4a7d-942b-48f916d18849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:00.565671 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd39a2c-03f8-4a7d-942b-48f916d18849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:00.567085 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd39a2c-03f8-4a7d-942b-48f916d18849 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:00.576567 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1248/2482] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:00.841041 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8080f-88c4-42d6-b835-78d4f7c0e96d None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:00.842545 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-26f8080f-88c4-42d6-b835-78d4f7c0e96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:00.855107 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8080f-88c4-42d6-b835-78d4f7c0e96d None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:00.860011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:00.860648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:00.860992 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:00.861263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] PATH_INFO: `/v3/projects/2bd0a559530442a6aebd24d57babf444` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:00.862614 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bd0a559530442a6aebd24d57babf444)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:00.864744 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:00.912859 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bd0a559530442a6aebd24d57babf444', 'request_id': 'req-f4293e46-60b4-47c6-9a0c-135a2e12f027'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:00.918322 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bd0a559530442a6aebd24d57babf444', 'request_id': 'req-f4293e46-60b4-47c6-9a0c-135a2e12f027'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:00.925573 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bd0a559530442a6aebd24d57babf444', 'request_id': 'req-f4293e46-60b4-47c6-9a0c-135a2e12f027'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:00.936990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] The token cache is being invalidate because project 2bd0a559530442a6aebd24d57babf444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:49:00.937278 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:00.937927 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-f4293e46-60b4-47c6-9a0c-135a2e12f027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:00.939905 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1235/2483] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:49:00 2022] DELETE /identity/v3/projects/2bd0a559530442a6aebd24d57babf444 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:49:00.944674 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e359f-e473-455e-bf9c-70be64b3037c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:00.945009 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e359f-e473-455e-bf9c-70be64b3037c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:00.945284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e359f-e473-455e-bf9c-70be64b3037c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:00.965403 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-270e359f-e473-455e-bf9c-70be64b3037c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:00.967540 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-270e359f-e473-455e-bf9c-70be64b3037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:00.979774 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:01.029911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1249/2484] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:01.038753 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4c13-637c-4c12-81f9-16e2c5281b45 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:01.039044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4c13-637c-4c12-81f9-16e2c5281b45 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:01.039317 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4c13-637c-4c12-81f9-16e2c5281b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:01.058492 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-849c4c13-637c-4c12-81f9-16e2c5281b45 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:01.059676 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-849c4c13-637c-4c12-81f9-16e2c5281b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:01.068061 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1236/2485] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:01.072106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e359f-e473-455e-bf9c-70be64b3037c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:01.082577 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e359f-e473-455e-bf9c-70be64b3037c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:01.087074 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd7b73-2b85-4249-8f5f-8891f762d994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:01.087663 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd7b73-2b85-4249-8f5f-8891f762d994 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:01.087939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd7b73-2b85-4249-8f5f-8891f762d994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:01.088210 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd7b73-2b85-4249-8f5f-8891f762d994 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:01.088924 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd7b73-2b85-4249-8f5f-8891f762d994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:49:01.089261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd7b73-2b85-4249-8f5f-8891f762d994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:01.090848 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd7b73-2b85-4249-8f5f-8891f762d994 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:01.101977 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1250/2486] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:49:01.106121 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c4c13-637c-4c12-81f9-16e2c5281b45 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:01.115176 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c4c13-637c-4c12-81f9-16e2c5281b45 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:01.119986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a506803-4a7a-410b-886a-a3b07da24fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:01.120581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a506803-4a7a-410b-886a-a3b07da24fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:01.120882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a506803-4a7a-410b-886a-a3b07da24fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:01.121192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a506803-4a7a-410b-886a-a3b07da24fa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:01.121812 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a506803-4a7a-410b-886a-a3b07da24fa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:01.123713 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a506803-4a7a-410b-886a-a3b07da24fa6 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:01.150775 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1237/2487] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:49:01 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:01.154826 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd7b73-2b85-4249-8f5f-8891f762d994 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:01.164564 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd7b73-2b85-4249-8f5f-8891f762d994 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:01.220108 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904ad20-e92c-4333-9e12-93678df4098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:01.220710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904ad20-e92c-4333-9e12-93678df4098e admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:01.221015 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904ad20-e92c-4333-9e12-93678df4098e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:01.221284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904ad20-e92c-4333-9e12-93678df4098e admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:01.221900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904ad20-e92c-4333-9e12-93678df4098e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:01.223781 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904ad20-e92c-4333-9e12-93678df4098e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:01.244972 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1251/2488] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:49:01 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:01.249480 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a506803-4a7a-410b-886a-a3b07da24fa6 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:01.256620 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:01.260514 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a506803-4a7a-410b-886a-a3b07da24fa6 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:01.266581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e786808-c270-485a-8bd6-254eb1ab4670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:01.267214 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786808-c270-485a-8bd6-254eb1ab4670 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:01.267539 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786808-c270-485a-8bd6-254eb1ab4670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:01.267813 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786808-c270-485a-8bd6-254eb1ab4670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:01.268455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e786808-c270-485a-8bd6-254eb1ab4670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:01.270100 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e786808-c270-485a-8bd6-254eb1ab4670 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:01.281081 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1238/2489] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:01.284984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904ad20-e92c-4333-9e12-93678df4098e admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:01.294662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904ad20-e92c-4333-9e12-93678df4098e admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:01.299789 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce644d2b-cdbe-423f-8858-e06e93146935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:01.300373 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce644d2b-cdbe-423f-8858-e06e93146935 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:01.300647 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce644d2b-cdbe-423f-8858-e06e93146935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:01.300969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce644d2b-cdbe-423f-8858-e06e93146935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:01.301601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce644d2b-cdbe-423f-8858-e06e93146935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:01.303170 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce644d2b-cdbe-423f-8858-e06e93146935 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:01.313655 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1252/2490] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:01.317960 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e786808-c270-485a-8bd6-254eb1ab4670 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:01.328327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e786808-c270-485a-8bd6-254eb1ab4670 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:01.333087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8d614-e729-4739-b316-4f133f521996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:01.333833 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8d614-e729-4739-b316-4f133f521996 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:01.334178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8d614-e729-4739-b316-4f133f521996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:01.334523 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8d614-e729-4739-b316-4f133f521996 admin admin] PATH_INFO: `/v3/projects/4148b7689d7c47f58db95b542570b7e6/users/0391499864c140ee971a9e6f6872edb8/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:01.350404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8d614-e729-4739-b316-4f133f521996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4148b7689d7c47f58db95b542570b7e6, user_id=0391499864c140ee971a9e6f6872edb8, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:01.353277 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8d614-e729-4739-b316-4f133f521996 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:01.366766 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1239/2491] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:01 2022] PUT /identity/v3/projects/4148b7689d7c47f58db95b542570b7e6/users/0391499864c140ee971a9e6f6872edb8/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:01.371959 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b8986-f10e-49b5-b58a-2776f83b069f None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:01.372326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b8986-f10e-49b5-b58a-2776f83b069f None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:01.372616 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b8986-f10e-49b5-b58a-2776f83b069f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:01.394326 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-951b8986-f10e-49b5-b58a-2776f83b069f None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:01.395541 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-951b8986-f10e-49b5-b58a-2776f83b069f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:01.456629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1253/2492] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:49:01.466329 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8d614-e729-4739-b316-4f133f521996 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:01.468087 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-aee8d614-e729-4739-b316-4f133f521996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:01.526342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8d614-e729-4739-b316-4f133f521996 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:01.530295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b33e7-d852-463a-818c-3f9f19b68a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:01.530913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b33e7-d852-463a-818c-3f9f19b68a4e service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:01.531192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b33e7-d852-463a-818c-3f9f19b68a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:01.531462 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b33e7-d852-463a-818c-3f9f19b68a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:01.541135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b33e7-d852-463a-818c-3f9f19b68a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:01.542601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b33e7-d852-463a-818c-3f9f19b68a4e service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:01.552595 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1240/2493] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:02.177724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009340f-05f2-43ed-85b6-e69095e1bfad None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:02.178072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009340f-05f2-43ed-85b6-e69095e1bfad None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:02.178292 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009340f-05f2-43ed-85b6-e69095e1bfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:02.207115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-f009340f-05f2-43ed-85b6-e69095e1bfad None None] MFA Rules not processed for user `0391499864c140ee971a9e6f6872edb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:02.208390 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f009340f-05f2-43ed-85b6-e69095e1bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:02.220491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:02.275545 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1254/2494] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:02.286680 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b33e7-d852-463a-818c-3f9f19b68a4e service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:02.288095 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-796b33e7-d852-463a-818c-3f9f19b68a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:02.367837 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b33e7-d852-463a-818c-3f9f19b68a4e service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:02.371547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1037852-885a-4280-a311-842ae3b27c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:02.372151 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037852-885a-4280-a311-842ae3b27c39 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:02.372433 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037852-885a-4280-a311-842ae3b27c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:02.372703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037852-885a-4280-a311-842ae3b27c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:02.381848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1037852-885a-4280-a311-842ae3b27c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:02.383280 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1037852-885a-4280-a311-842ae3b27c39 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:02.392171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1241/2495] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:03.021060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009340f-05f2-43ed-85b6-e69095e1bfad None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:03.022440 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f009340f-05f2-43ed-85b6-e69095e1bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:03.036032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009340f-05f2-43ed-85b6-e69095e1bfad None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:03.040462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:03.041105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:03.041383 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:03.041651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] PATH_INFO: `/v3/users/0aa68926bfc443958b8b5e954883237b` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:03.042753 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa68926bfc443958b8b5e954883237b)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:03.044635 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:03.088780 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa68926bfc443958b8b5e954883237b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.100948 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa68926bfc443958b8b5e954883237b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.101639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa68926bfc443958b8b5e954883237b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.108614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa68926bfc443958b8b5e954883237b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.109262 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa68926bfc443958b8b5e954883237b'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.121543 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1255/2496] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:49:03 2022] DELETE /identity/v3/users/0aa68926bfc443958b8b5e954883237b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:03.126135 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314e1fa-4e6f-4ff0-8aa8-17d10a894758 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:03.126627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314e1fa-4e6f-4ff0-8aa8-17d10a894758 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:03.126767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314e1fa-4e6f-4ff0-8aa8-17d10a894758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:03.145811 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7314e1fa-4e6f-4ff0-8aa8-17d10a894758 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:03.147008 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7314e1fa-4e6f-4ff0-8aa8-17d10a894758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:03.197243 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:03.210343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1242/2497] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:03.219875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:03.221299 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:03.277911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083fedeb-25f5-4847-92c8-810a6ed554b8 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:03.281631 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3041cb-816e-4617-8742-c87f37919dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:03.282238 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3041cb-816e-4617-8742-c87f37919dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:03.282523 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3041cb-816e-4617-8742-c87f37919dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:03.282798 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3041cb-816e-4617-8742-c87f37919dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:03.291928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3041cb-816e-4617-8742-c87f37919dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:03.293394 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3041cb-816e-4617-8742-c87f37919dd3 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:03.302839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1256/2498] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:03.567238 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314e1fa-4e6f-4ff0-8aa8-17d10a894758 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:03.568718 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7314e1fa-4e6f-4ff0-8aa8-17d10a894758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:03.586053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314e1fa-4e6f-4ff0-8aa8-17d10a894758 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:03.590888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:03.591657 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:03.591792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:03.592152 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] PATH_INFO: `/v3/projects/6e4630d42ccd4fd8b4f8e38b234e58f6` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:03.593206 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4630d42ccd4fd8b4f8e38b234e58f6)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:03.595123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:03.640965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4630d42ccd4fd8b4f8e38b234e58f6', 'request_id': 'req-4e6d8a16-1104-4fce-a42d-fc7c35243be5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.646326 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4630d42ccd4fd8b4f8e38b234e58f6', 'request_id': 'req-4e6d8a16-1104-4fce-a42d-fc7c35243be5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.653256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4630d42ccd4fd8b4f8e38b234e58f6', 'request_id': 'req-4e6d8a16-1104-4fce-a42d-fc7c35243be5'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.664130 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] The token cache is being invalidate because project 6e4630d42ccd4fd8b4f8e38b234e58f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:49:03.664418 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.665069 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4e6d8a16-1104-4fce-a42d-fc7c35243be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.666530 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1243/2499] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:49:03 2022] DELETE /identity/v3/projects/6e4630d42ccd4fd8b4f8e38b234e58f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:03.670447 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3041cb-816e-4617-8742-c87f37919dd3 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:03.680376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3041cb-816e-4617-8742-c87f37919dd3 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:03.706587 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:03.741266 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:03.741969 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:03.742265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:03.742536 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] PATH_INFO: `/v3/users/a450bb86d50a49638421147e2f2d9276` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:03.743640 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a450bb86d50a49638421147e2f2d9276)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:03.745639 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:03.756922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200a144-7c4f-40de-8403-e4f961d58199 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:03.757208 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200a144-7c4f-40de-8403-e4f961d58199 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:03.757485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200a144-7c4f-40de-8403-e4f961d58199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:03.777287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-4200a144-7c4f-40de-8403-e4f961d58199 None None] MFA Rules not processed for user `0391499864c140ee971a9e6f6872edb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:03.778506 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4200a144-7c4f-40de-8403-e4f961d58199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:03.794724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a450bb86d50a49638421147e2f2d9276'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.806583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a450bb86d50a49638421147e2f2d9276'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.807255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a450bb86d50a49638421147e2f2d9276'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.814426 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a450bb86d50a49638421147e2f2d9276'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.815095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a450bb86d50a49638421147e2f2d9276'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:03.828509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1257/2500] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:49:03 2022] DELETE /identity/v3/users/a450bb86d50a49638421147e2f2d9276 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:03.834809 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1244/2501] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:49:04.231579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:04.233073 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:04.300685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5611af5-a3b4-4eca-96e3-b1b056b7fb5f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:04.305341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:04.305971 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:04.306251 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:04.306525 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] PATH_INFO: `/v3/projects/38c187f4805b48329a017d8e43cc302b` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:04.307673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c187f4805b48329a017d8e43cc302b)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:04.309642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:04.359765 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c187f4805b48329a017d8e43cc302b', 'request_id': 'req-e16931b8-d44d-4766-bd9e-48c71702c533'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:04.361557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:04.367113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c187f4805b48329a017d8e43cc302b', 'request_id': 'req-e16931b8-d44d-4766-bd9e-48c71702c533'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:04.375870 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c187f4805b48329a017d8e43cc302b', 'request_id': 'req-e16931b8-d44d-4766-bd9e-48c71702c533'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:04.388766 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] The token cache is being invalidate because project 38c187f4805b48329a017d8e43cc302b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:49:04.389182 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:04.389911 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-e16931b8-d44d-4766-bd9e-48c71702c533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:04.391473 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1258/2502] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:49:04 2022] DELETE /identity/v3/projects/38c187f4805b48329a017d8e43cc302b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:04.396330 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b213d7-936b-4431-ba2b-3046562f61c2 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:04.396822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b213d7-936b-4431-ba2b-3046562f61c2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:04.396962 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b213d7-936b-4431-ba2b-3046562f61c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:04.416518 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-68b213d7-936b-4431-ba2b-3046562f61c2 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:04.417752 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-68b213d7-936b-4431-ba2b-3046562f61c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:04.484889 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1245/2503] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:49:04.490579 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464acb8f-a0f1-41f4-9d30-a75d26257d4d None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:04.490871 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464acb8f-a0f1-41f4-9d30-a75d26257d4d None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:04.491141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464acb8f-a0f1-41f4-9d30-a75d26257d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:04.512484 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-464acb8f-a0f1-41f4-9d30-a75d26257d4d None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:04.513852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-464acb8f-a0f1-41f4-9d30-a75d26257d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:04.522818 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1259/2504] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:04.529083 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b213d7-936b-4431-ba2b-3046562f61c2 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:04.538873 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b213d7-936b-4431-ba2b-3046562f61c2 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:04.543557 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8484a-5905-418c-b2b2-2132ac9e5bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:04.544158 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8484a-5905-418c-b2b2-2132ac9e5bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:04.544431 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8484a-5905-418c-b2b2-2132ac9e5bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:04.544697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8484a-5905-418c-b2b2-2132ac9e5bb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:04.545374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8484a-5905-418c-b2b2-2132ac9e5bb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:49:04.545673 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8484a-5905-418c-b2b2-2132ac9e5bb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:04.547266 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8484a-5905-418c-b2b2-2132ac9e5bb5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:04.561469 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1246/2505] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:49:04.566268 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464acb8f-a0f1-41f4-9d30-a75d26257d4d None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:04.575722 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464acb8f-a0f1-41f4-9d30-a75d26257d4d None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:04.580496 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceab6c2-ee97-4c07-9278-d5a5166cc3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:04.581425 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceab6c2-ee97-4c07-9278-d5a5166cc3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:04.581700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceab6c2-ee97-4c07-9278-d5a5166cc3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:04.581966 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceab6c2-ee97-4c07-9278-d5a5166cc3d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:04.582583 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceab6c2-ee97-4c07-9278-d5a5166cc3d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:04.584456 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceab6c2-ee97-4c07-9278-d5a5166cc3d6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:04.613021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1260/2506] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:49:04 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:04.617263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8484a-5905-418c-b2b2-2132ac9e5bb5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:04.628158 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8484a-5905-418c-b2b2-2132ac9e5bb5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:04.636105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:04.687212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d45627-d26f-49c3-b23a-f14ae8360060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:04.687742 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d45627-d26f-49c3-b23a-f14ae8360060 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:04.688023 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d45627-d26f-49c3-b23a-f14ae8360060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:04.688295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d45627-d26f-49c3-b23a-f14ae8360060 admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:04.688953 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d45627-d26f-49c3-b23a-f14ae8360060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:04.690884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d45627-d26f-49c3-b23a-f14ae8360060 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:04.714219 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1247/2507] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:49:04 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:04.718391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceab6c2-ee97-4c07-9278-d5a5166cc3d6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:04.729872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceab6c2-ee97-4c07-9278-d5a5166cc3d6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:04.734738 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff520838-adf7-4716-9753-c39c1963798d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:04.735341 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff520838-adf7-4716-9753-c39c1963798d admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:04.735620 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff520838-adf7-4716-9753-c39c1963798d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:04.736060 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff520838-adf7-4716-9753-c39c1963798d admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:04.737257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff520838-adf7-4716-9753-c39c1963798d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:04.738850 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff520838-adf7-4716-9753-c39c1963798d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:04.755289 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1261/2508] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:49:04.759598 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d45627-d26f-49c3-b23a-f14ae8360060 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:04.770171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d45627-d26f-49c3-b23a-f14ae8360060 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:04.774976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ee240-2fd8-4155-8cf9-30c829b8a15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:04.775586 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ee240-2fd8-4155-8cf9-30c829b8a15d admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:04.775857 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ee240-2fd8-4155-8cf9-30c829b8a15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:04.776129 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ee240-2fd8-4155-8cf9-30c829b8a15d admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:04.776769 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ee240-2fd8-4155-8cf9-30c829b8a15d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:04.778396 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ee240-2fd8-4155-8cf9-30c829b8a15d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:04.789975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1248/2509] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:04.794252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff520838-adf7-4716-9753-c39c1963798d admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:04.807455 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff520838-adf7-4716-9753-c39c1963798d admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:04.812651 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15437300-e8c9-4f3f-a173-4d8bc1cf651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:04.813349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15437300-e8c9-4f3f-a173-4d8bc1cf651f admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:04.813659 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15437300-e8c9-4f3f-a173-4d8bc1cf651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:04.813932 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15437300-e8c9-4f3f-a173-4d8bc1cf651f admin admin] PATH_INFO: `/v3/projects/39d66c7e13314d6e91f03cfb5eac35c2/users/3f05dba934a246598c47e7dbfce0b6a0/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:04.829863 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15437300-e8c9-4f3f-a173-4d8bc1cf651f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d66c7e13314d6e91f03cfb5eac35c2, user_id=3f05dba934a246598c47e7dbfce0b6a0, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:04.832494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15437300-e8c9-4f3f-a173-4d8bc1cf651f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:04.845084 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1262/2510] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:04 2022] PUT /identity/v3/projects/39d66c7e13314d6e91f03cfb5eac35c2/users/3f05dba934a246598c47e7dbfce0b6a0/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:04.857404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4879974-a456-45a4-8e86-3ca4ae542a2c None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:04.857731 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4879974-a456-45a4-8e86-3ca4ae542a2c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:04.858013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4879974-a456-45a4-8e86-3ca4ae542a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:04.886028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-d4879974-a456-45a4-8e86-3ca4ae542a2c None None] MFA Rules not processed for user `3f05dba934a246598c47e7dbfce0b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:04.887299 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-d4879974-a456-45a4-8e86-3ca4ae542a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:04.956382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1249/2511] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:49:04.961119 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15437300-e8c9-4f3f-a173-4d8bc1cf651f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:05.026521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15437300-e8c9-4f3f-a173-4d8bc1cf651f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:05.031242 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675a726-21a2-4b4d-8417-18f33fe7eb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:05.031858 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675a726-21a2-4b4d-8417-18f33fe7eb66 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:05.032135 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675a726-21a2-4b4d-8417-18f33fe7eb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:05.032403 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675a726-21a2-4b4d-8417-18f33fe7eb66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:05.033678 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0675a726-21a2-4b4d-8417-18f33fe7eb66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:05.035573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0675a726-21a2-4b4d-8417-18f33fe7eb66 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:05.061501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1263/2512] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:49:04 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:49:05.065677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4879974-a456-45a4-8e86-3ca4ae542a2c None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:05.077243 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4879974-a456-45a4-8e86-3ca4ae542a2c None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:05.135110 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a489b7-1dae-47a2-83bc-bfb7075b9f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:05.135721 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a489b7-1dae-47a2-83bc-bfb7075b9f2e admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:05.136000 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a489b7-1dae-47a2-83bc-bfb7075b9f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:05.136271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a489b7-1dae-47a2-83bc-bfb7075b9f2e admin admin] PATH_INFO: `/v3/users` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:05.137053 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a489b7-1dae-47a2-83bc-bfb7075b9f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:05.138836 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a489b7-1dae-47a2-83bc-bfb7075b9f2e admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:05.176271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1250/2513] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:49:05 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:49:05.177285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675a726-21a2-4b4d-8417-18f33fe7eb66 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:05.188391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675a726-21a2-4b4d-8417-18f33fe7eb66 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:05.193755 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ac13-9199-44c6-9b3c-aaff7f1fadec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:05.194490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ac13-9199-44c6-9b3c-aaff7f1fadec admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:05.195013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ac13-9199-44c6-9b3c-aaff7f1fadec admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:05.195403 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ac13-9199-44c6-9b3c-aaff7f1fadec admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:05.196070 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4ac13-9199-44c6-9b3c-aaff7f1fadec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:05.197724 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4ac13-9199-44c6-9b3c-aaff7f1fadec admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:05.209728 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1264/2514] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:05.213807 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a489b7-1dae-47a2-83bc-bfb7075b9f2e admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:05.224615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a489b7-1dae-47a2-83bc-bfb7075b9f2e admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:05.229366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5875755-8519-437d-bcd5-62c47ed9c326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:05.230015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5875755-8519-437d-bcd5-62c47ed9c326 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:05.230290 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5875755-8519-437d-bcd5-62c47ed9c326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:05.230558 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5875755-8519-437d-bcd5-62c47ed9c326 admin admin] PATH_INFO: `/v3/projects/3a1889e0648a40ab9d894d033e63f57e/users/015b40290f3d4fd0ab9d9366fef0a2d4/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:05.233605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:05.246352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5875755-8519-437d-bcd5-62c47ed9c326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1889e0648a40ab9d894d033e63f57e, user_id=015b40290f3d4fd0ab9d9366fef0a2d4, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:05.249050 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5875755-8519-437d-bcd5-62c47ed9c326 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:05.262257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1251/2515] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:05 2022] PUT /identity/v3/projects/3a1889e0648a40ab9d894d033e63f57e/users/015b40290f3d4fd0ab9d9366fef0a2d4/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:05.266152 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ac13-9199-44c6-9b3c-aaff7f1fadec admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:05.276516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ac13-9199-44c6-9b3c-aaff7f1fadec admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:05.319481 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:05.332889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd8c50-48d5-4890-9063-3d19d13ffa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:05.333499 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd8c50-48d5-4890-9063-3d19d13ffa1e admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:05.333778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd8c50-48d5-4890-9063-3d19d13ffa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:05.334051 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd8c50-48d5-4890-9063-3d19d13ffa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:05.334683 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd8c50-48d5-4890-9063-3d19d13ffa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:05.336310 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd8c50-48d5-4890-9063-3d19d13ffa1e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:05.346438 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1265/2516] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:05.350517 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5875755-8519-437d-bcd5-62c47ed9c326 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:05.360497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5875755-8519-437d-bcd5-62c47ed9c326 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:05.365039 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa5a2d-5b64-4030-a9f0-4131ca76f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:05.365677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa5a2d-5b64-4030-a9f0-4131ca76f600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:05.365949 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa5a2d-5b64-4030-a9f0-4131ca76f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:05.366217 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa5a2d-5b64-4030-a9f0-4131ca76f600 admin admin] PATH_INFO: `/v3/projects/3a1889e0648a40ab9d894d033e63f57e/users/015b40290f3d4fd0ab9d9366fef0a2d4/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:05.368103 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa5a2d-5b64-4030-a9f0-4131ca76f600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1889e0648a40ab9d894d033e63f57e, user_id=015b40290f3d4fd0ab9d9366fef0a2d4, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:05.370728 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa5a2d-5b64-4030-a9f0-4131ca76f600 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:05.380554 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1252/2517] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:05 2022] PUT /identity/v3/projects/3a1889e0648a40ab9d894d033e63f57e/users/015b40290f3d4fd0ab9d9366fef0a2d4/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:05.390359 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59cdbd-4951-4444-84c4-21c19e3b7612 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:05.390654 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59cdbd-4951-4444-84c4-21c19e3b7612 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:05.390936 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59cdbd-4951-4444-84c4-21c19e3b7612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:05.419323 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-9f59cdbd-4951-4444-84c4-21c19e3b7612 None None] MFA Rules not processed for user `015b40290f3d4fd0ab9d9366fef0a2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:05.420535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9f59cdbd-4951-4444-84c4-21c19e3b7612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:05.487896 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1266/2518] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:05.499890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa5a2d-5b64-4030-a9f0-4131ca76f600 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:05.502235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-66fa5a2d-5b64-4030-a9f0-4131ca76f600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:05.568203 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa5a2d-5b64-4030-a9f0-4131ca76f600 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:05.572193 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5f737-cd97-41ca-8a30-91541d7908c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:05.572822 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5f737-cd97-41ca-8a30-91541d7908c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:05.573111 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5f737-cd97-41ca-8a30-91541d7908c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:05.573379 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5f737-cd97-41ca-8a30-91541d7908c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:05.585121 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5f737-cd97-41ca-8a30-91541d7908c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:05.586792 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5f737-cd97-41ca-8a30-91541d7908c5 service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:05.598307 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1253/2519] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:05.942257 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59cdbd-4951-4444-84c4-21c19e3b7612 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:05.951842 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59cdbd-4951-4444-84c4-21c19e3b7612 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:05.957528 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab77c3d-03af-4c61-961c-5d53714e0b48 tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '015b40290f3d4fd0ab9d9366fef0a2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a1889e0648a40ab9d894d033e63f57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:05.958160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab77c3d-03af-4c61-961c-5d53714e0b48 tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:05.958549 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab77c3d-03af-4c61-961c-5d53714e0b48 tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:05.958832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab77c3d-03af-4c61-961c-5d53714e0b48 tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:05.959491 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab77c3d-03af-4c61-961c-5d53714e0b48 tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:49:05.959796 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab77c3d-03af-4c61-961c-5d53714e0b48 tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:05.961413 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab77c3d-03af-4c61-961c-5d53714e0b48 tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:05.974064 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1267/2520] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:49:05.978323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5f737-cd97-41ca-8a30-91541d7908c5 service cinder] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:05.988477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5f737-cd97-41ca-8a30-91541d7908c5 service cinder] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:05.993790 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520647a0-a493-4bbc-a4f3-c7e425d6b79f tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '015b40290f3d4fd0ab9d9366fef0a2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a1889e0648a40ab9d894d033e63f57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:05.994331 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520647a0-a493-4bbc-a4f3-c7e425d6b79f tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:05.994629 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520647a0-a493-4bbc-a4f3-c7e425d6b79f tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:05.994900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520647a0-a493-4bbc-a4f3-c7e425d6b79f tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:05.995566 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520647a0-a493-4bbc-a4f3-c7e425d6b79f tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:05.997489 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520647a0-a493-4bbc-a4f3-c7e425d6b79f tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:06.030369 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1254/2521] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:49:05 2022] POST /identity/v3/projects => generated 354 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:06.032209 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab77c3d-03af-4c61-961c-5d53714e0b48 tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:06.042531 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab77c3d-03af-4c61-961c-5d53714e0b48 tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:06.099997 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c488c7a-b752-486a-b6d9-3186a65dc51d tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '015b40290f3d4fd0ab9d9366fef0a2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a1889e0648a40ab9d894d033e63f57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:06.100715 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c488c7a-b752-486a-b6d9-3186a65dc51d tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:06.101106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c488c7a-b752-486a-b6d9-3186a65dc51d tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:06.101445 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c488c7a-b752-486a-b6d9-3186a65dc51d tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:06.102157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c488c7a-b752-486a-b6d9-3186a65dc51d tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:49:06.102568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c488c7a-b752-486a-b6d9-3186a65dc51d tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:06.104190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c488c7a-b752-486a-b6d9-3186a65dc51d tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:06.116405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1268/2522] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:49:06.153171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520647a0-a493-4bbc-a4f3-c7e425d6b79f tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:06.164238 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520647a0-a493-4bbc-a4f3-c7e425d6b79f tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:06.169332 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '015b40290f3d4fd0ab9d9366fef0a2d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a1889e0648a40ab9d894d033e63f57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:06.170116 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:06.170496 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:06.170868 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] PATH_INFO: `/v3/projects/a2d1625d45464a3fa805b84bd60e0e7d` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:06.172106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a2d1625d45464a3fa805b84bd60e0e7d)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:06.174192 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:06.230646 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d1625d45464a3fa805b84bd60e0e7d', 'request_id': 'req-7a95edac-7dee-4846-b80b-bd79b91c2d8b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:06.236150 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d1625d45464a3fa805b84bd60e0e7d', 'request_id': 'req-7a95edac-7dee-4846-b80b-bd79b91c2d8b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:06.243975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d1625d45464a3fa805b84bd60e0e7d', 'request_id': 'req-7a95edac-7dee-4846-b80b-bd79b91c2d8b'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:06.257198 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] The token cache is being invalidate because project a2d1625d45464a3fa805b84bd60e0e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:49:06.257327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:06.257939 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-7a95edac-7dee-4846-b80b-bd79b91c2d8b tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:06.259644 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1255/2523] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:49:06 2022] DELETE /identity/v3/projects/a2d1625d45464a3fa805b84bd60e0e7d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:49:06.458227 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c488c7a-b752-486a-b6d9-3186a65dc51d tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:06.460159 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9c488c7a-b752-486a-b6d9-3186a65dc51d tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:06.510927 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=100 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:06.525314 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c488c7a-b752-486a-b6d9-3186a65dc51d tempest-VolumeQuotaClassesTest-2115245592 tempest-VolumeQuotaClassesTest-2115245592-project-admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:06.583136 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:06.583769 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:06.584045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:06.584313 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] PATH_INFO: `/v3/users/3f05dba934a246598c47e7dbfce0b6a0` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:06.585461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f05dba934a246598c47e7dbfce0b6a0)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:06.587354 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:06.632544 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f05dba934a246598c47e7dbfce0b6a0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:06.645102 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f05dba934a246598c47e7dbfce0b6a0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:06.645829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f05dba934a246598c47e7dbfce0b6a0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:06.653370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f05dba934a246598c47e7dbfce0b6a0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:06.654205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f05dba934a246598c47e7dbfce0b6a0'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:06.667409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1269/2524] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:49:06 2022] DELETE /identity/v3/users/3f05dba934a246598c47e7dbfce0b6a0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:06.671864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90519c3a-57d4-4475-952b-6976c3a018fc None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:06.672368 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90519c3a-57d4-4475-952b-6976c3a018fc None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:06.672777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90519c3a-57d4-4475-952b-6976c3a018fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:06.696874 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-90519c3a-57d4-4475-952b-6976c3a018fc None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:06.698170 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-90519c3a-57d4-4475-952b-6976c3a018fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:06.746363 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:06.779221 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1256/2525] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:06.792019 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:06.793811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:06.861399 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc02281-7fcb-4e5f-870f-be17a824d57f admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:06.865217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcc397-0cc0-4f21-b41c-7cf2a5ef8e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:06.865823 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcc397-0cc0-4f21-b41c-7cf2a5ef8e2b service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:06.866103 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcc397-0cc0-4f21-b41c-7cf2a5ef8e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:06.866366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcc397-0cc0-4f21-b41c-7cf2a5ef8e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:06.879106 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcc397-0cc0-4f21-b41c-7cf2a5ef8e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:06.880747 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcc397-0cc0-4f21-b41c-7cf2a5ef8e2b service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:06.890612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1270/2526] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:07.568056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90519c3a-57d4-4475-952b-6976c3a018fc None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:07.569594 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-90519c3a-57d4-4475-952b-6976c3a018fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:07.583360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90519c3a-57d4-4475-952b-6976c3a018fc None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:07.588229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:07.588996 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:07.589159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:07.589432 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] PATH_INFO: `/v3/projects/39d66c7e13314d6e91f03cfb5eac35c2` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:07.590562 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39d66c7e13314d6e91f03cfb5eac35c2)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:07.592534 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:07.645134 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39d66c7e13314d6e91f03cfb5eac35c2', 'request_id': 'req-e41311fc-5033-409c-b7b4-b08a729893e0'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:07.651026 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39d66c7e13314d6e91f03cfb5eac35c2', 'request_id': 'req-e41311fc-5033-409c-b7b4-b08a729893e0'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:07.659183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39d66c7e13314d6e91f03cfb5eac35c2', 'request_id': 'req-e41311fc-5033-409c-b7b4-b08a729893e0'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:07.670447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] The token cache is being invalidate because project 39d66c7e13314d6e91f03cfb5eac35c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:49:07.670734 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:07.671370 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:07.672905 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1257/2527] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:49:07 2022] DELETE /identity/v3/projects/39d66c7e13314d6e91f03cfb5eac35c2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:07.676949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcc397-0cc0-4f21-b41c-7cf2a5ef8e2b service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:07.686850 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcc397-0cc0-4f21-b41c-7cf2a5ef8e2b service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:07.705315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:07.746333 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:07.746964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:07.747243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:07.747542 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] PATH_INFO: `/v3/users/015b40290f3d4fd0ab9d9366fef0a2d4` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:07.748706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=015b40290f3d4fd0ab9d9366fef0a2d4)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:07.750648 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:07.795436 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '015b40290f3d4fd0ab9d9366fef0a2d4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:07.807259 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '015b40290f3d4fd0ab9d9366fef0a2d4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:07.807939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '015b40290f3d4fd0ab9d9366fef0a2d4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:07.815643 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '015b40290f3d4fd0ab9d9366fef0a2d4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:07.816209 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8ed9396a-ee6b-44d3-87fa-871fef52cdd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '015b40290f3d4fd0ab9d9366fef0a2d4'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:07.828906 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1271/2528] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:49:07 2022] DELETE /identity/v3/users/015b40290f3d4fd0ab9d9366fef0a2d4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:08.172320 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:08.174471 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:08.194698 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41311fc-5033-409c-b7b4-b08a729893e0 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:08.253025 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:08.253661 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.253968 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.254259 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] PATH_INFO: `/v3/projects/3a1889e0648a40ab9d894d033e63f57e` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.255364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a1889e0648a40ab9d894d033e63f57e)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:08.257304 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:08.303056 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a1889e0648a40ab9d894d033e63f57e', 'request_id': 'req-71488606-ef1a-452a-8d62-40ee3026bd3c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:08.308540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a1889e0648a40ab9d894d033e63f57e', 'request_id': 'req-71488606-ef1a-452a-8d62-40ee3026bd3c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:08.310965 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:08.316010 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a1889e0648a40ab9d894d033e63f57e', 'request_id': 'req-71488606-ef1a-452a-8d62-40ee3026bd3c'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:08.328460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] The token cache is being invalidate because project 3a1889e0648a40ab9d894d033e63f57e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:49:08.328748 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:08.329460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-71488606-ef1a-452a-8d62-40ee3026bd3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:08.331490 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1258/2529] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:49:08 2022] DELETE /identity/v3/projects/3a1889e0648a40ab9d894d033e63f57e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:08.336568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337802cd-21bc-4051-bedb-b05bef284e70 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.336880 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337802cd-21bc-4051-bedb-b05bef284e70 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.337166 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337802cd-21bc-4051-bedb-b05bef284e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.357281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-337802cd-21bc-4051-bedb-b05bef284e70 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:08.358421 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-337802cd-21bc-4051-bedb-b05bef284e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:08.416750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1272/2530] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:08.424560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd09d1-5c3f-430f-a10f-e1ccada624fc None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.424864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd09d1-5c3f-430f-a10f-e1ccada624fc None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.425154 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd09d1-5c3f-430f-a10f-e1ccada624fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.444798 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-09fd09d1-5c3f-430f-a10f-e1ccada624fc None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:08.446029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-09fd09d1-5c3f-430f-a10f-e1ccada624fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:08.454119 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1259/2531] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:08.460214 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337802cd-21bc-4051-bedb-b05bef284e70 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:08.469841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337802cd-21bc-4051-bedb-b05bef284e70 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:08.474315 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0454b-1970-4f11-bd3c-631566490d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:08.474905 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0454b-1970-4f11-bd3c-631566490d76 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.475180 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0454b-1970-4f11-bd3c-631566490d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.475452 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0454b-1970-4f11-bd3c-631566490d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.476081 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0454b-1970-4f11-bd3c-631566490d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:49:08.476494 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0454b-1970-4f11-bd3c-631566490d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:08.478113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0454b-1970-4f11-bd3c-631566490d76 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:08.489199 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1273/2532] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 18 11:49:08.493263 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd09d1-5c3f-430f-a10f-e1ccada624fc None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:08.502738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd09d1-5c3f-430f-a10f-e1ccada624fc None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:08.507250 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb05e7c-401c-413e-a94f-7c65b01bcf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:08.507834 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb05e7c-401c-413e-a94f-7c65b01bcf98 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.508106 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb05e7c-401c-413e-a94f-7c65b01bcf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.508372 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb05e7c-401c-413e-a94f-7c65b01bcf98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.509035 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb05e7c-401c-413e-a94f-7c65b01bcf98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:08.510904 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb05e7c-401c-413e-a94f-7c65b01bcf98 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:08.538141 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1260/2533] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:49:08 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:08.542283 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0454b-1970-4f11-bd3c-631566490d76 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:08.552254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0454b-1970-4f11-bd3c-631566490d76 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:08.585296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:08.621907 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9f590-0767-4120-8f2b-1b14da090dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:08.622963 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9f590-0767-4120-8f2b-1b14da090dbd admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.623442 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9f590-0767-4120-8f2b-1b14da090dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.623938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9f590-0767-4120-8f2b-1b14da090dbd admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.624930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9f590-0767-4120-8f2b-1b14da090dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:08.627054 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9f590-0767-4120-8f2b-1b14da090dbd admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:08.653934 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1274/2534] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:49:08 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:08.658284 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb05e7c-401c-413e-a94f-7c65b01bcf98 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:08.668677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb05e7c-401c-413e-a94f-7c65b01bcf98 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:08.674342 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ef06e-7c04-4e80-bfe2-351d89389328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:08.674987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ef06e-7c04-4e80-bfe2-351d89389328 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.674987 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ef06e-7c04-4e80-bfe2-351d89389328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.675156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ef06e-7c04-4e80-bfe2-351d89389328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.676117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ef06e-7c04-4e80-bfe2-351d89389328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:08.677417 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ef06e-7c04-4e80-bfe2-351d89389328 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:08.691526 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1261/2535] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:08.692736 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9f590-0767-4120-8f2b-1b14da090dbd admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:08.705032 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9f590-0767-4120-8f2b-1b14da090dbd admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:08.710653 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1bb50-17a1-4f56-add7-9f48c447a19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:08.711514 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1bb50-17a1-4f56-add7-9f48c447a19d admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.711868 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1bb50-17a1-4f56-add7-9f48c447a19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.713358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1bb50-17a1-4f56-add7-9f48c447a19d admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.713358 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1bb50-17a1-4f56-add7-9f48c447a19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:08.714642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1bb50-17a1-4f56-add7-9f48c447a19d admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:08.726573 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1275/2536] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:49:08.731173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ef06e-7c04-4e80-bfe2-351d89389328 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:08.743225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ef06e-7c04-4e80-bfe2-351d89389328 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:08.748466 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8ce17-a3ec-4b2c-90ad-a273fbdddd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:08.749212 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8ce17-a3ec-4b2c-90ad-a273fbdddd8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.749579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8ce17-a3ec-4b2c-90ad-a273fbdddd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.749861 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8ce17-a3ec-4b2c-90ad-a273fbdddd8b admin admin] PATH_INFO: `/v3/projects/a30521e63a174c3b9bfc29546263c9cf/users/137da23e8baf417c88e83737ca8e9a43/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.766456 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8ce17-a3ec-4b2c-90ad-a273fbdddd8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a30521e63a174c3b9bfc29546263c9cf, user_id=137da23e8baf417c88e83737ca8e9a43, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:08.770188 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8ce17-a3ec-4b2c-90ad-a273fbdddd8b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:08.785705 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1262/2537] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:08 2022] PUT /identity/v3/projects/a30521e63a174c3b9bfc29546263c9cf/users/137da23e8baf417c88e83737ca8e9a43/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:08.796141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815de1e-5f37-4df9-9f3b-5f0941850b6c None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.796432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815de1e-5f37-4df9-9f3b-5f0941850b6c None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.796710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815de1e-5f37-4df9-9f3b-5f0941850b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.828548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-e815de1e-5f37-4df9-9f3b-5f0941850b6c None None] MFA Rules not processed for user `137da23e8baf417c88e83737ca8e9a43`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:08.830389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-e815de1e-5f37-4df9-9f3b-5f0941850b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:08.896347 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1276/2538] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:08.900808 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8ce17-a3ec-4b2c-90ad-a273fbdddd8b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:08.965460 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8ce17-a3ec-4b2c-90ad-a273fbdddd8b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:08.970281 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38346f6-ace2-4f11-9b26-eddb5e976e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:08.970882 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38346f6-ace2-4f11-9b26-eddb5e976e0b admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:08.971155 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38346f6-ace2-4f11-9b26-eddb5e976e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:08.971424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38346f6-ace2-4f11-9b26-eddb5e976e0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:08.972063 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38346f6-ace2-4f11-9b26-eddb5e976e0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:08.974483 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38346f6-ace2-4f11-9b26-eddb5e976e0b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:09.000720 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1263/2539] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:49:08 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:09.004943 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815de1e-5f37-4df9-9f3b-5f0941850b6c None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:09.015895 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815de1e-5f37-4df9-9f3b-5f0941850b6c None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:09.074118 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a411e8-6706-4c73-ac9d-77ac1cf2cfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:09.074729 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a411e8-6706-4c73-ac9d-77ac1cf2cfa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:09.075007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a411e8-6706-4c73-ac9d-77ac1cf2cfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:09.075281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a411e8-6706-4c73-ac9d-77ac1cf2cfa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:09.075904 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a411e8-6706-4c73-ac9d-77ac1cf2cfa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:09.077835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a411e8-6706-4c73-ac9d-77ac1cf2cfa2 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:09.101457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1277/2540] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:49:09 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:09.103234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38346f6-ace2-4f11-9b26-eddb5e976e0b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:09.113584 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38346f6-ace2-4f11-9b26-eddb5e976e0b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:09.118292 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f534715-5ced-4c94-aa53-b702debf5745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:09.118891 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f534715-5ced-4c94-aa53-b702debf5745 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:09.119171 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f534715-5ced-4c94-aa53-b702debf5745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:09.119448 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f534715-5ced-4c94-aa53-b702debf5745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:09.120093 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f534715-5ced-4c94-aa53-b702debf5745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:09.121733 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f534715-5ced-4c94-aa53-b702debf5745 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:09.137654 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1264/2541] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:09.138998 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a411e8-6706-4c73-ac9d-77ac1cf2cfa2 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:09.151742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a411e8-6706-4c73-ac9d-77ac1cf2cfa2 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:09.156516 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6635d-1aff-4726-ace8-b791bf7f91f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:09.157217 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6635d-1aff-4726-ace8-b791bf7f91f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:09.157497 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6635d-1aff-4726-ace8-b791bf7f91f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:09.158352 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6635d-1aff-4726-ace8-b791bf7f91f1 admin admin] PATH_INFO: `/v3/projects/3277ca7e5cfe40ee95c9b3141e43fa68/users/cf8539e98d9f40849327f81da92c3346/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:09.187401 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6635d-1aff-4726-ace8-b791bf7f91f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3277ca7e5cfe40ee95c9b3141e43fa68, user_id=cf8539e98d9f40849327f81da92c3346, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:09.191008 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6635d-1aff-4726-ace8-b791bf7f91f1 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:09.202627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:09.208889 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1278/2542] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:09 2022] PUT /identity/v3/projects/3277ca7e5cfe40ee95c9b3141e43fa68/users/cf8539e98d9f40849327f81da92c3346/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:09.212937 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f534715-5ced-4c94-aa53-b702debf5745 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:09.223159 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f534715-5ced-4c94-aa53-b702debf5745 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:09.287455 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f0ec7-1728-46c5-96ed-28e9f8d27962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:09.288072 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0ec7-1728-46c5-96ed-28e9f8d27962 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:09.288352 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0ec7-1728-46c5-96ed-28e9f8d27962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:09.288633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f0ec7-1728-46c5-96ed-28e9f8d27962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:09.289301 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f0ec7-1728-46c5-96ed-28e9f8d27962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:09.290942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f0ec7-1728-46c5-96ed-28e9f8d27962 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:09.292298 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:09.308862 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1265/2543] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:09.310069 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6635d-1aff-4726-ace8-b791bf7f91f1 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:09.320742 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6635d-1aff-4726-ace8-b791bf7f91f1 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:09.325608 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c521b8f-f239-4885-b724-e484a808a311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:09.326249 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521b8f-f239-4885-b724-e484a808a311 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:09.326524 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521b8f-f239-4885-b724-e484a808a311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:09.326792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c521b8f-f239-4885-b724-e484a808a311 admin admin] PATH_INFO: `/v3/projects/3277ca7e5cfe40ee95c9b3141e43fa68/users/cf8539e98d9f40849327f81da92c3346/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:09.328673 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c521b8f-f239-4885-b724-e484a808a311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3277ca7e5cfe40ee95c9b3141e43fa68, user_id=cf8539e98d9f40849327f81da92c3346, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:09.331847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c521b8f-f239-4885-b724-e484a808a311 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:09.341664 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1279/2544] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:09 2022] PUT /identity/v3/projects/3277ca7e5cfe40ee95c9b3141e43fa68/users/cf8539e98d9f40849327f81da92c3346/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:09.351828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad47686-3b4d-40e2-b036-8debfd099479 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:09.352117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad47686-3b4d-40e2-b036-8debfd099479 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:09.352393 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad47686-3b4d-40e2-b036-8debfd099479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:09.379928 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-bad47686-3b4d-40e2-b036-8debfd099479 None None] MFA Rules not processed for user `cf8539e98d9f40849327f81da92c3346`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:09.381166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-bad47686-3b4d-40e2-b036-8debfd099479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:09.449526 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1266/2545] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:09.460535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c521b8f-f239-4885-b724-e484a808a311 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:09.462264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-3c521b8f-f239-4885-b724-e484a808a311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:09.518837 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c521b8f-f239-4885-b724-e484a808a311 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:09.522553 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3556d7f-9439-4ea4-bb94-33ef6abd1156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:09.523145 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3556d7f-9439-4ea4-bb94-33ef6abd1156 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:09.523421 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3556d7f-9439-4ea4-bb94-33ef6abd1156 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:09.523689 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3556d7f-9439-4ea4-bb94-33ef6abd1156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:09.532641 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3556d7f-9439-4ea4-bb94-33ef6abd1156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:09.534162 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3556d7f-9439-4ea4-bb94-33ef6abd1156 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:09.543389 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1280/2546] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:09.645348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad47686-3b4d-40e2-b036-8debfd099479 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:09.654975 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad47686-3b4d-40e2-b036-8debfd099479 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:09.658547 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e621f6-8981-4554-9c5c-292c68ee1d9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:09.659137 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e621f6-8981-4554-9c5c-292c68ee1d9c service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:09.659409 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e621f6-8981-4554-9c5c-292c68ee1d9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:09.659677 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e621f6-8981-4554-9c5c-292c68ee1d9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:09.715613 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e621f6-8981-4554-9c5c-292c68ee1d9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:09.717015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e621f6-8981-4554-9c5c-292c68ee1d9c service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:09.725863 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1267/2547] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:12.298835 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3556d7f-9439-4ea4-bb94-33ef6abd1156 service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:12.359065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3556d7f-9439-4ea4-bb94-33ef6abd1156 service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:12.364362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:12.365021 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:12.365305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:12.365578 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] PATH_INFO: `/v3/users/137da23e8baf417c88e83737ca8e9a43` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:12.366800 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=137da23e8baf417c88e83737ca8e9a43)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:12.369795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:12.415013 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '137da23e8baf417c88e83737ca8e9a43'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:12.427289 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '137da23e8baf417c88e83737ca8e9a43'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:12.427994 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '137da23e8baf417c88e83737ca8e9a43'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:12.435443 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '137da23e8baf417c88e83737ca8e9a43'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:12.436235 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '137da23e8baf417c88e83737ca8e9a43'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:12.449582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1281/2548] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:49:12 2022] DELETE /identity/v3/users/137da23e8baf417c88e83737ca8e9a43 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:12.454091 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f94c8-347c-472b-82a3-c55defb8e4b7 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:12.454758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f94c8-347c-472b-82a3-c55defb8e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:12.454758 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f94c8-347c-472b-82a3-c55defb8e4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:12.473610 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-995f94c8-347c-472b-82a3-c55defb8e4b7 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:12.474903 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-995f94c8-347c-472b-82a3-c55defb8e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:12.536055 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1268/2549] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:12.545557 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:12.547001 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:12.558512 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:12.605778 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f1f03-c39c-41be-b2e0-20a992913dca admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:12.609418 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa0a6c-b636-4133-8043-b7faea7bd661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:12.610018 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa0a6c-b636-4133-8043-b7faea7bd661 service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:12.610297 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa0a6c-b636-4133-8043-b7faea7bd661 service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:12.610568 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa0a6c-b636-4133-8043-b7faea7bd661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:12.619888 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa0a6c-b636-4133-8043-b7faea7bd661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:12.621336 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa0a6c-b636-4133-8043-b7faea7bd661 service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:12.630478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1282/2550] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:12.922488 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f94c8-347c-472b-82a3-c55defb8e4b7 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:12.923867 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-995f94c8-347c-472b-82a3-c55defb8e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:12.937253 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f94c8-347c-472b-82a3-c55defb8e4b7 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:12.942605 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:12.943348 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:12.943719 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:12.944173 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] PATH_INFO: `/v3/projects/a30521e63a174c3b9bfc29546263c9cf` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:12.945684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a30521e63a174c3b9bfc29546263c9cf)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:12.948295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:12.971140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=100 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:13.003323 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a30521e63a174c3b9bfc29546263c9cf', 'request_id': 'req-cecbf968-e14c-4f98-9a0f-b251e6185d26'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.009195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a30521e63a174c3b9bfc29546263c9cf', 'request_id': 'req-cecbf968-e14c-4f98-9a0f-b251e6185d26'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.016890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a30521e63a174c3b9bfc29546263c9cf', 'request_id': 'req-cecbf968-e14c-4f98-9a0f-b251e6185d26'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.029497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] The token cache is being invalidate because project a30521e63a174c3b9bfc29546263c9cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:49:13.029788 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.030446 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.031946 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1269/2551] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:49:12 2022] DELETE /identity/v3/projects/a30521e63a174c3b9bfc29546263c9cf => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:13.036432 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa0a6c-b636-4133-8043-b7faea7bd661 service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:13.046065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa0a6c-b636-4133-8043-b7faea7bd661 service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:13.102851 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:13.103519 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:13.103792 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:13.104062 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] PATH_INFO: `/v3/users/cf8539e98d9f40849327f81da92c3346` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:13.105219 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf8539e98d9f40849327f81da92c3346)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:13.107113 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:13.154349 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf8539e98d9f40849327f81da92c3346'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.169595 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf8539e98d9f40849327f81da92c3346'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.170406 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf8539e98d9f40849327f81da92c3346'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.178101 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf8539e98d9f40849327f81da92c3346'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.178776 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-d313c21a-1dae-47ac-9aed-3a16320d5f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf8539e98d9f40849327f81da92c3346'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.193753 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1283/2552] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:49:13 2022] DELETE /identity/v3/users/cf8539e98d9f40849327f81da92c3346 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:13.473394 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:13.475287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:13.489697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbf968-e14c-4f98-9a0f-b251e6185d26 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:13.551038 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:13.551684 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:13.551955 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:13.552224 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] PATH_INFO: `/v3/projects/3277ca7e5cfe40ee95c9b3141e43fa68` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:13.553367 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3277ca7e5cfe40ee95c9b3141e43fa68)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:13.555300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:13.614177 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3277ca7e5cfe40ee95c9b3141e43fa68', 'request_id': 'req-5eec1da0-d2db-495b-b063-c18453e4eeb6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.620073 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3277ca7e5cfe40ee95c9b3141e43fa68', 'request_id': 'req-5eec1da0-d2db-495b-b063-c18453e4eeb6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.629067 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3277ca7e5cfe40ee95c9b3141e43fa68', 'request_id': 'req-5eec1da0-d2db-495b-b063-c18453e4eeb6'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.641704 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] The token cache is being invalidate because project 3277ca7e5cfe40ee95c9b3141e43fa68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:49:13.641994 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.642652 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-5eec1da0-d2db-495b-b063-c18453e4eeb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:13.644637 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1270/2553] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:49:13 2022] DELETE /identity/v3/projects/3277ca7e5cfe40ee95c9b3141e43fa68 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:13.649730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0697f-3656-420c-b325-639042d51f18 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:13.650254 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0697f-3656-420c-b325-639042d51f18 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:13.650362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0697f-3656-420c-b325-639042d51f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:13.670366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-92b0697f-3656-420c-b325-639042d51f18 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:13.671570 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-92b0697f-3656-420c-b325-639042d51f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:13.697629 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:13.734711 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1284/2554] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:13.797497 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245315cd-dc28-4d6b-a80c-bff269c6991c None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:13.797784 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245315cd-dc28-4d6b-a80c-bff269c6991c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:13.798064 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245315cd-dc28-4d6b-a80c-bff269c6991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:13.818398 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-245315cd-dc28-4d6b-a80c-bff269c6991c None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:13.819524 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-245315cd-dc28-4d6b-a80c-bff269c6991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:13.827714 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1271/2555] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:13.832535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0697f-3656-420c-b325-639042d51f18 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:13.842168 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0697f-3656-420c-b325-639042d51f18 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:13.847282 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832502b-e5dc-4ae0-9d41-84bec6b9d0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:13.847903 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832502b-e5dc-4ae0-9d41-84bec6b9d0ca admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:13.848205 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832502b-e5dc-4ae0-9d41-84bec6b9d0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:13.848475 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832502b-e5dc-4ae0-9d41-84bec6b9d0ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:13.849164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832502b-e5dc-4ae0-9d41-84bec6b9d0ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:49:13.849461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832502b-e5dc-4ae0-9d41-84bec6b9d0ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:13.851033 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832502b-e5dc-4ae0-9d41-84bec6b9d0ca admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:13.863335 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1285/2556] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:49:13.867740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245315cd-dc28-4d6b-a80c-bff269c6991c None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:13.877269 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245315cd-dc28-4d6b-a80c-bff269c6991c None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:13.882119 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26c139-134b-42c6-909b-b141cadb3ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:13.882712 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26c139-134b-42c6-909b-b141cadb3ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:13.882986 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26c139-134b-42c6-909b-b141cadb3ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:13.883262 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26c139-134b-42c6-909b-b141cadb3ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:13.883888 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26c139-134b-42c6-909b-b141cadb3ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:13.885844 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26c139-134b-42c6-909b-b141cadb3ab5 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:13.913879 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1272/2557] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:49:13 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:13.918255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832502b-e5dc-4ae0-9d41-84bec6b9d0ca admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:13.930022 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832502b-e5dc-4ae0-9d41-84bec6b9d0ca admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:13.987288 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b285c55-b6d2-4690-adb7-5344c0a2f91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:13.988140 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b285c55-b6d2-4690-adb7-5344c0a2f91e admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:13.988532 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b285c55-b6d2-4690-adb7-5344c0a2f91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:13.988930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b285c55-b6d2-4690-adb7-5344c0a2f91e admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:13.989759 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b285c55-b6d2-4690-adb7-5344c0a2f91e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:13.992235 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b285c55-b6d2-4690-adb7-5344c0a2f91e admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:14.021457 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1286/2558] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:49:13 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:14.025473 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26c139-134b-42c6-909b-b141cadb3ab5 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:14.032145 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:14.036886 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26c139-134b-42c6-909b-b141cadb3ab5 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:14.042969 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee71ce5-c41a-4708-a877-acedbee92059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:14.043862 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71ce5-c41a-4708-a877-acedbee92059 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:14.044371 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71ce5-c41a-4708-a877-acedbee92059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:14.044816 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee71ce5-c41a-4708-a877-acedbee92059 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:14.045537 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee71ce5-c41a-4708-a877-acedbee92059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:14.047580 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee71ce5-c41a-4708-a877-acedbee92059 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:14.057682 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1273/2559] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:14.061732 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b285c55-b6d2-4690-adb7-5344c0a2f91e admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:14.072706 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b285c55-b6d2-4690-adb7-5344c0a2f91e admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:14.077771 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63a007-3499-48b4-87aa-ec0497647a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:14.078385 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63a007-3499-48b4-87aa-ec0497647a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:14.078665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63a007-3499-48b4-87aa-ec0497647a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:14.078938 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63a007-3499-48b4-87aa-ec0497647a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:14.079582 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63a007-3499-48b4-87aa-ec0497647a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:14.081222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63a007-3499-48b4-87aa-ec0497647a09 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:14.092623 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1287/2560] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:14.096780 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee71ce5-c41a-4708-a877-acedbee92059 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:14.106701 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee71ce5-c41a-4708-a877-acedbee92059 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:14.111645 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2bbef-2a65-4731-bec7-50c6edc132bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:14.112314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2bbef-2a65-4731-bec7-50c6edc132bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:14.112603 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2bbef-2a65-4731-bec7-50c6edc132bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:14.112896 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2bbef-2a65-4731-bec7-50c6edc132bd admin admin] PATH_INFO: `/v3/projects/1a726faad00a4ecd8032c32d6c38ffc3/users/b0a1e99f9e7d40cb830764ca5fbc0cd5/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:14.127163 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2bbef-2a65-4731-bec7-50c6edc132bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a726faad00a4ecd8032c32d6c38ffc3, user_id=b0a1e99f9e7d40cb830764ca5fbc0cd5, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:14.130391 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2bbef-2a65-4731-bec7-50c6edc132bd admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:14.146531 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1274/2561] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:14 2022] PUT /identity/v3/projects/1a726faad00a4ecd8032c32d6c38ffc3/users/b0a1e99f9e7d40cb830764ca5fbc0cd5/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:14.156470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a438a3-a72f-41af-88e6-46d982ac168a None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:14.156758 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a438a3-a72f-41af-88e6-46d982ac168a None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:14.157065 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a438a3-a72f-41af-88e6-46d982ac168a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:14.190538 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-a3a438a3-a72f-41af-88e6-46d982ac168a None None] MFA Rules not processed for user `b0a1e99f9e7d40cb830764ca5fbc0cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:14.192279 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-a3a438a3-a72f-41af-88e6-46d982ac168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:14.257220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1288/2562] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:14.270223 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2bbef-2a65-4731-bec7-50c6edc132bd admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:14.271947 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-05c2bbef-2a65-4731-bec7-50c6edc132bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:14.328549 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2bbef-2a65-4731-bec7-50c6edc132bd admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:14.339429 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5a311-4ea3-4620-b6e8-b31bfb81ef0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:14.340084 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5a311-4ea3-4620-b6e8-b31bfb81ef0a service cinder] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:14.340387 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5a311-4ea3-4620-b6e8-b31bfb81ef0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:14.340700 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5a311-4ea3-4620-b6e8-b31bfb81ef0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:14.351374 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5a311-4ea3-4620-b6e8-b31bfb81ef0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:14.352828 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5a311-4ea3-4620-b6e8-b31bfb81ef0a service cinder] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:14.361831 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1275/2563] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:22.780937 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a438a3-a72f-41af-88e6-46d982ac168a None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:22.810160 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:22.845811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a438a3-a72f-41af-88e6-46d982ac168a None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:22.850956 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:22.851585 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:22.851866 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:22.852139 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] PATH_INFO: `/v3/users/b0a1e99f9e7d40cb830764ca5fbc0cd5` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:22.853298 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a1e99f9e7d40cb830764ca5fbc0cd5)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:22.855235 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:22.900376 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a1e99f9e7d40cb830764ca5fbc0cd5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:22.911985 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a1e99f9e7d40cb830764ca5fbc0cd5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:22.913261 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a1e99f9e7d40cb830764ca5fbc0cd5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:22.919928 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a1e99f9e7d40cb830764ca5fbc0cd5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:22.920548 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a1e99f9e7d40cb830764ca5fbc0cd5'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:22.933440 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1289/2564] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:49:22 2022] DELETE /identity/v3/users/b0a1e99f9e7d40cb830764ca5fbc0cd5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:22.938051 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb82616-eb55-4fe9-beab-112cd0cc8528 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:22.938540 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb82616-eb55-4fe9-beab-112cd0cc8528 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:22.938668 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb82616-eb55-4fe9-beab-112cd0cc8528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:22.958123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-8cb82616-eb55-4fe9-beab-112cd0cc8528 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:22.959926 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8cb82616-eb55-4fe9-beab-112cd0cc8528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:23.016525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1276/2565] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:23.027105 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:23.028296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:23.085472 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05c99d-173e-4e13-b650-be4a5982b931 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:23.089190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb53bcc-942f-43d7-b94b-d3425d47957d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:23.089846 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb53bcc-942f-43d7-b94b-d3425d47957d service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.090127 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb53bcc-942f-43d7-b94b-d3425d47957d service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.090397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb53bcc-942f-43d7-b94b-d3425d47957d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.099308 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb53bcc-942f-43d7-b94b-d3425d47957d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:23.100716 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb53bcc-942f-43d7-b94b-d3425d47957d service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:23.110059 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1290/2566] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:23.376973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb82616-eb55-4fe9-beab-112cd0cc8528 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:23.378407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-8cb82616-eb55-4fe9-beab-112cd0cc8528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:23.390835 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb82616-eb55-4fe9-beab-112cd0cc8528 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:23.395643 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:23.396274 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.396572 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.396991 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] PATH_INFO: `/v3/projects/1a726faad00a4ecd8032c32d6c38ffc3` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.398014 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a726faad00a4ecd8032c32d6c38ffc3)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:23.399940 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:23.446897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a726faad00a4ecd8032c32d6c38ffc3', 'request_id': 'req-8d5357d4-5433-4543-8022-de23a4e68eff'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:23.452591 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a726faad00a4ecd8032c32d6c38ffc3', 'request_id': 'req-8d5357d4-5433-4543-8022-de23a4e68eff'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:23.459457 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a726faad00a4ecd8032c32d6c38ffc3', 'request_id': 'req-8d5357d4-5433-4543-8022-de23a4e68eff'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:23.461360 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:23.472702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] The token cache is being invalidate because project 1a726faad00a4ecd8032c32d6c38ffc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:49:23.473028 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:23.473661 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-8d5357d4-5433-4543-8022-de23a4e68eff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:49:23.475164 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1277/2567] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:49:23 2022] DELETE /identity/v3/projects/1a726faad00a4ecd8032c32d6c38ffc3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:23.480053 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5977f3-30fb-4909-9288-fc0eadb51270 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.480366 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5977f3-30fb-4909-9288-fc0eadb51270 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.480665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5977f3-30fb-4909-9288-fc0eadb51270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.501222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-9e5977f3-30fb-4909-9288-fc0eadb51270 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:23.502614 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-9e5977f3-30fb-4909-9288-fc0eadb51270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:23.547832 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:23.567392 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1291/2568] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 18 11:49:23.575913 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819be85-8ed3-4e30-b7be-4920013435c5 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.576228 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819be85-8ed3-4e30-b7be-4920013435c5 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.576525 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819be85-8ed3-4e30-b7be-4920013435c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.596105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-7819be85-8ed3-4e30-b7be-4920013435c5 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:23.597367 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-7819be85-8ed3-4e30-b7be-4920013435c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:23.605229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1278/2569] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:23.609408 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5977f3-30fb-4909-9288-fc0eadb51270 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:23.619461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5977f3-30fb-4909-9288-fc0eadb51270 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:23.624007 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f3c58-b884-4159-a4c6-babb5f0e6de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:23.624603 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3c58-b884-4159-a4c6-babb5f0e6de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.624914 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3c58-b884-4159-a4c6-babb5f0e6de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.625193 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3c58-b884-4159-a4c6-babb5f0e6de4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.625841 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f3c58-b884-4159-a4c6-babb5f0e6de4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 18 11:49:23.626144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f3c58-b884-4159-a4c6-babb5f0e6de4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:23.627760 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f3c58-b884-4159-a4c6-babb5f0e6de4 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:23.639606 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1292/2570] 10.222.0.17 () {58 vars in 1153 bytes} [Tue Oct 18 11:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 18 11:49:23.643944 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819be85-8ed3-4e30-b7be-4920013435c5 None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:23.653312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819be85-8ed3-4e30-b7be-4920013435c5 None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:23.657990 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b86ea3-fa5f-4a23-86c7-2275b74c847b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:23.658579 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b86ea3-fa5f-4a23-86c7-2275b74c847b admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.658854 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b86ea3-fa5f-4a23-86c7-2275b74c847b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.659123 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b86ea3-fa5f-4a23-86c7-2275b74c847b admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.659754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b86ea3-fa5f-4a23-86c7-2275b74c847b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:23.661685 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b86ea3-fa5f-4a23-86c7-2275b74c847b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:23.688364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1279/2571] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:49:23 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:23.692470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f3c58-b884-4159-a4c6-babb5f0e6de4 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:23.703045 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f3c58-b884-4159-a4c6-babb5f0e6de4 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:23.758872 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090b8d2-d626-42ea-a9c0-8598bbc121a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:23.759468 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090b8d2-d626-42ea-a9c0-8598bbc121a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.759763 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090b8d2-d626-42ea-a9c0-8598bbc121a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.760020 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090b8d2-d626-42ea-a9c0-8598bbc121a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.760649 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090b8d2-d626-42ea-a9c0-8598bbc121a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:23.762594 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090b8d2-d626-42ea-a9c0-8598bbc121a0 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:23.783986 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1293/2572] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:49:23 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:23.788225 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b86ea3-fa5f-4a23-86c7-2275b74c847b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:23.798542 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b86ea3-fa5f-4a23-86c7-2275b74c847b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:23.803470 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02843e55-82fc-4fe2-adc1-5f9b4a322aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:23.804079 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02843e55-82fc-4fe2-adc1-5f9b4a322aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.804359 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02843e55-82fc-4fe2-adc1-5f9b4a322aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.804633 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02843e55-82fc-4fe2-adc1-5f9b4a322aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.805316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02843e55-82fc-4fe2-adc1-5f9b4a322aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:23.807044 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02843e55-82fc-4fe2-adc1-5f9b4a322aa7 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:23.817376 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1280/2573] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:23.821726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090b8d2-d626-42ea-a9c0-8598bbc121a0 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:23.831921 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090b8d2-d626-42ea-a9c0-8598bbc121a0 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:23.837346 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56a8da-07d5-4240-8926-26fe549589fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:23.838087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56a8da-07d5-4240-8926-26fe549589fa admin admin] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.838281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56a8da-07d5-4240-8926-26fe549589fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.838560 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56a8da-07d5-4240-8926-26fe549589fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.839255 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56a8da-07d5-4240-8926-26fe549589fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:23.840833 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56a8da-07d5-4240-8926-26fe549589fa admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:23.851559 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1294/2574] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:23.855781 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02843e55-82fc-4fe2-adc1-5f9b4a322aa7 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:23.865576 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02843e55-82fc-4fe2-adc1-5f9b4a322aa7 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:23.870058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2eca5-e422-4012-9428-14a8d75cc60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:23.870693 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2eca5-e422-4012-9428-14a8d75cc60b admin admin] REQUEST_METHOD: `PUT` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.870966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2eca5-e422-4012-9428-14a8d75cc60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.871234 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2eca5-e422-4012-9428-14a8d75cc60b admin admin] PATH_INFO: `/v3/projects/b6ef293a199241bf94848b937058ced5/users/b69c66f5c4c34e0eb85975cd7d53a90c/roles/846f40e28e104ec4bc568fe533f9c9a9` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.885760 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2eca5-e422-4012-9428-14a8d75cc60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ef293a199241bf94848b937058ced5, user_id=b69c66f5c4c34e0eb85975cd7d53a90c, role_id=846f40e28e104ec4bc568fe533f9c9a9)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:23.888520 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2eca5-e422-4012-9428-14a8d75cc60b admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:23.901648 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1281/2575] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:23 2022] PUT /identity/v3/projects/b6ef293a199241bf94848b937058ced5/users/b69c66f5c4c34e0eb85975cd7d53a90c/roles/846f40e28e104ec4bc568fe533f9c9a9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:49:23.906807 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec39a6-d25f-4162-a672-94c9edcb8f73 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:23.907094 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec39a6-d25f-4162-a672-94c9edcb8f73 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:23.907370 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec39a6-d25f-4162-a672-94c9edcb8f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:23.925964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-95ec39a6-d25f-4162-a672-94c9edcb8f73 None None] MFA Rules not processed for user `c5f193bd980d483b8200f674f5249745`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:23.927164 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-95ec39a6-d25f-4162-a672-94c9edcb8f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:23.985224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1295/2576] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:23.994755 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2eca5-e422-4012-9428-14a8d75cc60b admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:23.996465 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e2c2eca5-e422-4012-9428-14a8d75cc60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:24.052627 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2eca5-e422-4012-9428-14a8d75cc60b admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:24.056966 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cc58a-d095-41f1-9fa9-de6cb83b319d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:24.057626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cc58a-d095-41f1-9fa9-de6cb83b319d service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:24.057907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cc58a-d095-41f1-9fa9-de6cb83b319d service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:24.058183 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cc58a-d095-41f1-9fa9-de6cb83b319d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:24.067174 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cc58a-d095-41f1-9fa9-de6cb83b319d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:24.069178 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cc58a-d095-41f1-9fa9-de6cb83b319d service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:24.078733 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1282/2577] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:49:27.983429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb4b4b-9928-4aff-9c09-a2960ef802bb None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:27.983964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb4b4b-9928-4aff-9c09-a2960ef802bb None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:27.983964 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb4b4b-9928-4aff-9c09-a2960ef802bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:28.013274 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-6bcb4b4b-9928-4aff-9c09-a2960ef802bb None None] MFA Rules not processed for user `b69c66f5c4c34e0eb85975cd7d53a90c`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:28.014490 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-6bcb4b4b-9928-4aff-9c09-a2960ef802bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:28.075120 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:28.114984 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cc58a-d095-41f1-9fa9-de6cb83b319d service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:28.115554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1296/2578] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:28.130132 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cc58a-d095-41f1-9fa9-de6cb83b319d service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:28.135519 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f5666-95fb-43f0-96e5-a02ae7d4fdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:28.136129 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f5666-95fb-43f0-96e5-a02ae7d4fdd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:28.136403 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f5666-95fb-43f0-96e5-a02ae7d4fdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:28.136663 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f5666-95fb-43f0-96e5-a02ae7d4fdd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:28.137337 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f5666-95fb-43f0-96e5-a02ae7d4fdd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:28.139235 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f5666-95fb-43f0-96e5-a02ae7d4fdd1 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:28.190247 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb4b4b-9928-4aff-9c09-a2960ef802bb None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:28.195124 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1283/2579] 10.222.0.17 () {60 vars in 1153 bytes} [Tue Oct 18 11:49:28 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 18 11:49:28.213004 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb4b4b-9928-4aff-9c09-a2960ef802bb None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:28.325661 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25648b-e319-4652-83df-cfd61c7c3729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:28.326556 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25648b-e319-4652-83df-cfd61c7c3729 admin admin] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:28.327012 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25648b-e319-4652-83df-cfd61c7c3729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:28.327462 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25648b-e319-4652-83df-cfd61c7c3729 admin admin] PATH_INFO: `/v3/users` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:28.328281 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25648b-e319-4652-83df-cfd61c7c3729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:28.330329 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25648b-e319-4652-83df-cfd61c7c3729 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:28.369719 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1297/2580] 10.222.0.17 () {60 vars in 1144 bytes} [Tue Oct 18 11:49:28 2022] POST /identity/v3/users => generated 296 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 18 11:49:28.373195 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f5666-95fb-43f0-96e5-a02ae7d4fdd1 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:28.392334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f5666-95fb-43f0-96e5-a02ae7d4fdd1 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:28.398973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc2c49-e84e-4467-9207-7706efe89d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:28.399800 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc2c49-e84e-4467-9207-7706efe89d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:28.400271 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc2c49-e84e-4467-9207-7706efe89d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:28.400709 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc2c49-e84e-4467-9207-7706efe89d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:28.401798 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc2c49-e84e-4467-9207-7706efe89d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:28.405381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc2c49-e84e-4467-9207-7706efe89d8d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:28.425373 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1284/2581] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 18 11:49:28.429730 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25648b-e319-4652-83df-cfd61c7c3729 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:28.447448 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25648b-e319-4652-83df-cfd61c7c3729 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:28.461500 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f921f1-a50c-4999-a6ec-8d436e31fc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:28.462296 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f921f1-a50c-4999-a6ec-8d436e31fc29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:28.462761 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f921f1-a50c-4999-a6ec-8d436e31fc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:28.462916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f921f1-a50c-4999-a6ec-8d436e31fc29 admin admin] PATH_INFO: `/v3/projects/56c1847e28f3482d8a2eeecfd5f32806/users/ccac288563cb4db8ade7094e6b95f61e/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:28.482195 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f921f1-a50c-4999-a6ec-8d436e31fc29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56c1847e28f3482d8a2eeecfd5f32806, user_id=ccac288563cb4db8ade7094e6b95f61e, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:28.485129 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f921f1-a50c-4999-a6ec-8d436e31fc29 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:28.518265 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1298/2582] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:28 2022] PUT /identity/v3/projects/56c1847e28f3482d8a2eeecfd5f32806/users/ccac288563cb4db8ade7094e6b95f61e/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:49:28.523426 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc2c49-e84e-4467-9207-7706efe89d8d admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:28.537207 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc2c49-e84e-4467-9207-7706efe89d8d admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:28.633138 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951cb48-4df8-4571-9515-a3bef75a111d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:28.633747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951cb48-4df8-4571-9515-a3bef75a111d admin admin] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:28.634043 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951cb48-4df8-4571-9515-a3bef75a111d admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:28.634314 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951cb48-4df8-4571-9515-a3bef75a111d admin admin] PATH_INFO: `/v3/roles` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:28.634951 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951cb48-4df8-4571-9515-a3bef75a111d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:28.636638 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951cb48-4df8-4571-9515-a3bef75a111d admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:28.639871 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:28.653560 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1285/2583] 10.222.0.17 () {58 vars in 1122 bytes} [Tue Oct 18 11:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 18 11:49:28.654723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f921f1-a50c-4999-a6ec-8d436e31fc29 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:28.665592 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f921f1-a50c-4999-a6ec-8d436e31fc29 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:28.670252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865171-4173-4e88-8592-cf8ff7b2ebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:28.670916 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865171-4173-4e88-8592-cf8ff7b2ebf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:28.671189 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865171-4173-4e88-8592-cf8ff7b2ebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:28.671459 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865171-4173-4e88-8592-cf8ff7b2ebf6 admin admin] PATH_INFO: `/v3/projects/56c1847e28f3482d8a2eeecfd5f32806/users/ccac288563cb4db8ade7094e6b95f61e/roles/8d9ff9da6ad946a6bc49336777182ff9` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:28.673454 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28865171-4173-4e88-8592-cf8ff7b2ebf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56c1847e28f3482d8a2eeecfd5f32806, user_id=ccac288563cb4db8ade7094e6b95f61e, role_id=8d9ff9da6ad946a6bc49336777182ff9)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:28.676087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28865171-4173-4e88-8592-cf8ff7b2ebf6 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:28.687232 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1299/2584] 10.222.0.17 () {60 vars in 1483 bytes} [Tue Oct 18 11:49:28 2022] PUT /identity/v3/projects/56c1847e28f3482d8a2eeecfd5f32806/users/ccac288563cb4db8ade7094e6b95f61e/roles/8d9ff9da6ad946a6bc49336777182ff9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:49:32.772703 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07747c01-038c-4dc6-b4f9-942c5f4b33d2 None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:32.773255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07747c01-038c-4dc6-b4f9-942c5f4b33d2 None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:32.773521 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07747c01-038c-4dc6-b4f9-942c5f4b33d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:32.840382 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-07747c01-038c-4dc6-b4f9-942c5f4b33d2 None None] MFA Rules not processed for user `ccac288563cb4db8ade7094e6b95f61e`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:32.841707 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-07747c01-038c-4dc6-b4f9-942c5f4b33d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:32.995365 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1286/2585] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:49:33.014410 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865171-4173-4e88-8592-cf8ff7b2ebf6 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:49:33.017097 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-28865171-4173-4e88-8592-cf8ff7b2ebf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:33.153521 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865171-4173-4e88-8592-cf8ff7b2ebf6 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:49:33.159360 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce829dc-07b3-44a4-9bc4-4f23893cb4ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:49:33.159981 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce829dc-07b3-44a4-9bc4-4f23893cb4ab service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:33.160264 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce829dc-07b3-44a4-9bc4-4f23893cb4ab service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:33.160539 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce829dc-07b3-44a4-9bc4-4f23893cb4ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:33.172086 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:33.233362 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce829dc-07b3-44a4-9bc4-4f23893cb4ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:49:33.234959 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce829dc-07b3-44a4-9bc4-4f23893cb4ab service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:49:33.244271 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1300/2586] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:49:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:49:42.991481 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=35 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:43.240169 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=24 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:49:51.971240 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175d7c1-e953-4edb-8014-7bf1d495290e None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:49:51.971756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175d7c1-e953-4edb-8014-7bf1d495290e None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:49:51.971756 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175d7c1-e953-4edb-8014-7bf1d495290e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:49:51.991204 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-9175d7c1-e953-4edb-8014-7bf1d495290e None None] MFA Rules not processed for user `b69c66f5c4c34e0eb85975cd7d53a90c`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:49:51.992414 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9175d7c1-e953-4edb-8014-7bf1d495290e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:49:51.999877 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1287/2587] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:49:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:50:01.986366 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:50:04.122172 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb24899-2a72-46a4-8236-8795c53f70b1 None None] REQUEST_METHOD: `POST` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:50:04.122633 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb24899-2a72-46a4-8236-8795c53f70b1 None None] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:50:04.122751 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb24899-2a72-46a4-8236-8795c53f70b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:50:04.141821 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.auth.core [None req-7cb24899-2a72-46a4-8236-8795c53f70b1 None None] MFA Rules not processed for user `0391499864c140ee971a9e6f6872edb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:50:04.142995 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7cb24899-2a72-46a4-8236-8795c53f70b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:50:04.197409 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1301/2588] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:50:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:50:11.875296 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175d7c1-e953-4edb-8014-7bf1d495290e None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:50:11.877616 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-9175d7c1-e953-4edb-8014-7bf1d495290e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:50:11.942334 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175d7c1-e953-4edb-8014-7bf1d495290e None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:50:11.946900 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a41371-b87c-4dd7-8c59-e3f05b76255c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:50:11.947559 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a41371-b87c-4dd7-8c59-e3f05b76255c service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:50:11.947860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a41371-b87c-4dd7-8c59-e3f05b76255c service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:50:11.948127 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a41371-b87c-4dd7-8c59-e3f05b76255c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:50:11.957942 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a41371-b87c-4dd7-8c59-e3f05b76255c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:50:11.959370 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a41371-b87c-4dd7-8c59-e3f05b76255c service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:50:11.969804 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1288/2589] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:50:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 18 11:50:14.190144 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=25 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:50:21.965256 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=26 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:50:32.977900 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb24899-2a72-46a4-8236-8795c53f70b1 None None] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:50:32.979284 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7cb24899-2a72-46a4-8236-8795c53f70b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:50:33.035030 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb24899-2a72-46a4-8236-8795c53f70b1 None None] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:50:33.038795 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a04fa-9154-48b1-ac87-e7091cb0e76a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:50:33.039430 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a04fa-9154-48b1-ac87-e7091cb0e76a service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:50:33.039710 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a04fa-9154-48b1-ac87-e7091cb0e76a service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:50:33.039982 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a04fa-9154-48b1-ac87-e7091cb0e76a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:50:33.041727 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8b9a04fa-9154-48b1-ac87-e7091cb0e76a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:50:33.054768 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a04fa-9154-48b1-ac87-e7091cb0e76a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:50:33.056694 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a04fa-9154-48b1-ac87-e7091cb0e76a service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:50:33.067537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1302/2590] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:50:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:50:35.665584 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a41371-b87c-4dd7-8c59-e3f05b76255c service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:50:35.674440 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a41371-b87c-4dd7-8c59-e3f05b76255c service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:50:35.678140 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d71851-b275-4451-9fc7-8b099a14a080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:50:35.678738 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d71851-b275-4451-9fc7-8b099a14a080 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:50:35.679013 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d71851-b275-4451-9fc7-8b099a14a080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:50:35.679287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d71851-b275-4451-9fc7-8b099a14a080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:50:35.681060 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-71d71851-b275-4451-9fc7-8b099a14a080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:50:35.694533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d71851-b275-4451-9fc7-8b099a14a080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:50:35.696697 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d71851-b275-4451-9fc7-8b099a14a080 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:50:35.707601 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1289/2591] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:50:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:50:43.062697 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=26 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:50:45.702692 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=6 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:51:00.941224 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a04fa-9154-48b1-ac87-e7091cb0e76a service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:51:00.942777 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8b9a04fa-9154-48b1-ac87-e7091cb0e76a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:51:01.000011 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d71851-b275-4451-9fc7-8b099a14a080 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:51:01.001485 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-71d71851-b275-4451-9fc7-8b099a14a080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:51:01.023537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a04fa-9154-48b1-ac87-e7091cb0e76a service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:51:01.029584 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b6063-f4e2-4d3f-abf7-5c3dd28bee40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:51:01.030188 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b6063-f4e2-4d3f-abf7-5c3dd28bee40 service placement] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:51:01.030470 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b6063-f4e2-4d3f-abf7-5c3dd28bee40 service placement] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:51:01.030740 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b6063-f4e2-4d3f-abf7-5c3dd28bee40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:51:01.032501 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-fd9b6063-f4e2-4d3f-abf7-5c3dd28bee40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:51:01.044111 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b6063-f4e2-4d3f-abf7-5c3dd28bee40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:51:01.045662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b6063-f4e2-4d3f-abf7-5c3dd28bee40 service placement] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:51:01.055642 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1303/2592] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:51:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:51:01.067450 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d71851-b275-4451-9fc7-8b099a14a080 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:51:01.072715 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:51:01.073381 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:51:01.074229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:51:01.074636 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] PATH_INFO: `/v3/users/0391499864c140ee971a9e6f6872edb8` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:51:01.075801 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0391499864c140ee971a9e6f6872edb8)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:51:01.077743 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:51:01.123468 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0391499864c140ee971a9e6f6872edb8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:51:01.130343 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0391499864c140ee971a9e6f6872edb8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:51:01.130995 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0391499864c140ee971a9e6f6872edb8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:51:01.143312 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0391499864c140ee971a9e6f6872edb8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:51:01.150407 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0391499864c140ee971a9e6f6872edb8'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:51:01.157419 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1290/2593] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:51:00 2022] DELETE /identity/v3/users/0391499864c140ee971a9e6f6872edb8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 18 11:51:01.325129 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b6063-f4e2-4d3f-abf7-5c3dd28bee40 service placement] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:51:01.337194 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b6063-f4e2-4d3f-abf7-5c3dd28bee40 service placement] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:51:01.394954 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:51:01.395580 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:51:01.395860 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:51:01.396138 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] PATH_INFO: `/v3/projects/4148b7689d7c47f58db95b542570b7e6` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:51:01.397270 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4148b7689d7c47f58db95b542570b7e6)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:51:01.399186 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:51:01.443369 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4148b7689d7c47f58db95b542570b7e6', 'request_id': 'req-8bd120f1-f362-445a-917e-ca1114d33774'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:51:01.449569 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4148b7689d7c47f58db95b542570b7e6', 'request_id': 'req-8bd120f1-f362-445a-917e-ca1114d33774'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:51:01.456605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4148b7689d7c47f58db95b542570b7e6', 'request_id': 'req-8bd120f1-f362-445a-917e-ca1114d33774'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:51:01.467527 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] The token cache is being invalidate because project 4148b7689d7c47f58db95b542570b7e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:51:01.467839 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:51:01.468451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:51:01.469937 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1304/2594] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:51:01 2022] DELETE /identity/v3/projects/4148b7689d7c47f58db95b542570b7e6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:51:11.153615 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:51:11.464605 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:51:35.490848 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:51:35.492316 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:51:35.547897 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453a695-37dd-4b8b-9201-bf4a17ddee15 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:51:35.594287 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ec75d-5a7e-4804-a121-0edc0635b6a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:51:35.594883 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec75d-5a7e-4804-a121-0edc0635b6a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:51:35.595156 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec75d-5a7e-4804-a121-0edc0635b6a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:51:35.595421 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec75d-5a7e-4804-a121-0edc0635b6a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:51:35.597142 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-e85ec75d-5a7e-4804-a121-0edc0635b6a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:51:35.606148 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ec75d-5a7e-4804-a121-0edc0635b6a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:51:35.608094 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ec75d-5a7e-4804-a121-0edc0635b6a9 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:51:35.615902 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1291/2595] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:51:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:51:45.612414 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=46 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:52:24.254700 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:52:24.257036 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:52:24.338133 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd120f1-f362-445a-917e-ca1114d33774 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:52:24.344263 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b17a0b-d9ee-4f22-88a5-8dd8235a69ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:52:24.345186 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b17a0b-d9ee-4f22-88a5-8dd8235a69ce service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:52:24.345652 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b17a0b-d9ee-4f22-88a5-8dd8235a69ce service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:52:24.346115 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b17a0b-d9ee-4f22-88a5-8dd8235a69ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:52:24.349102 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-31b17a0b-d9ee-4f22-88a5-8dd8235a69ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:52:24.363811 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b17a0b-d9ee-4f22-88a5-8dd8235a69ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:52:24.365478 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b17a0b-d9ee-4f22-88a5-8dd8235a69ce service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:52:24.377949 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1305/2596] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:52:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:52:25.130764 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a699b2c-237d-480e-99d0-1ac979e733de None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:52:25.131309 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a699b2c-237d-480e-99d0-1ac979e733de None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:52:25.131576 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a699b2c-237d-480e-99d0-1ac979e733de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:52:25.152524 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-2a699b2c-237d-480e-99d0-1ac979e733de None None] MFA Rules not processed for user `b69c66f5c4c34e0eb85975cd7d53a90c`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:52:25.153777 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2a699b2c-237d-480e-99d0-1ac979e733de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:52:25.221896 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1292/2597] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:52:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:52:25.550489 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b17a0b-d9ee-4f22-88a5-8dd8235a69ce service nova] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:52:25.551875 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-31b17a0b-d9ee-4f22-88a5-8dd8235a69ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:52:25.563098 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b17a0b-d9ee-4f22-88a5-8dd8235a69ce service nova] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:52:25.566662 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6198afa-b79d-4276-aade-4758505e24bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:52:25.567253 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6198afa-b79d-4276-aade-4758505e24bc service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:52:25.567537 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6198afa-b79d-4276-aade-4758505e24bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:52:25.567812 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6198afa-b79d-4276-aade-4758505e24bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:52:25.569535 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d6198afa-b79d-4276-aade-4758505e24bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:52:25.578996 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6198afa-b79d-4276-aade-4758505e24bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:52:25.580405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6198afa-b79d-4276-aade-4758505e24bc service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:52:25.588243 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1306/2598] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:52:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:52:26.033876 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a699b2c-237d-480e-99d0-1ac979e733de None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:52:26.035364 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2a699b2c-237d-480e-99d0-1ac979e733de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:52:26.107581 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a699b2c-237d-480e-99d0-1ac979e733de None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:52:26.113149 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b51cf6-da87-423a-94a3-1627533b5cec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:52:26.113884 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b51cf6-da87-423a-94a3-1627533b5cec service placement] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:52:26.114025 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b51cf6-da87-423a-94a3-1627533b5cec service placement] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:52:26.114295 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b51cf6-da87-423a-94a3-1627533b5cec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:52:26.116029 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-63b51cf6-da87-423a-94a3-1627533b5cec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:52:26.127420 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b51cf6-da87-423a-94a3-1627533b5cec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:52:26.129015 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b51cf6-da87-423a-94a3-1627533b5cec service placement] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:52:26.138745 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1293/2599] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:52:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:52:35.584726 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=40 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:52:36.134463 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=57 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:52:40.351618 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6198afa-b79d-4276-aade-4758505e24bc service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:52:40.353312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-d6198afa-b79d-4276-aade-4758505e24bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:52:40.370601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6198afa-b79d-4276-aade-4758505e24bc service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:52:40.376003 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d2b92-7517-4930-b1a4-56534dfc2552 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:52:40.377185 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d2b92-7517-4930-b1a4-56534dfc2552 service nova] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:52:40.377461 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d2b92-7517-4930-b1a4-56534dfc2552 service nova] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:52:40.377723 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d2b92-7517-4930-b1a4-56534dfc2552 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:52:40.379427 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f80d2b92-7517-4930-b1a4-56534dfc2552 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:52:40.391509 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d2b92-7517-4930-b1a4-56534dfc2552 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:52:40.393805 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d2b92-7517-4930-b1a4-56534dfc2552 service nova] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:52:40.404554 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1307/2600] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:52:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:52:44.426300 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b51cf6-da87-423a-94a3-1627533b5cec service placement] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:52:44.438624 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b51cf6-da87-423a-94a3-1627533b5cec service placement] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:52:44.444059 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878d000-1d3a-4420-b7bf-f429736ffa9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f8a4a8277745a9ad1976af9df31cb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:52:44.444664 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878d000-1d3a-4420-b7bf-f429736ffa9e service nova] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:52:44.444973 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878d000-1d3a-4420-b7bf-f429736ffa9e service nova] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:52:44.445248 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878d000-1d3a-4420-b7bf-f429736ffa9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:52:44.447002 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0878d000-1d3a-4420-b7bf-f429736ffa9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:52:44.459906 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878d000-1d3a-4420-b7bf-f429736ffa9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:52:44.461510 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878d000-1d3a-4420-b7bf-f429736ffa9e service nova] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:52:44.472922 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1294/2601] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:52:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:52:50.400132 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=6 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:52:54.467602 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=6 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:53:37.381474 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d2b92-7517-4930-b1a4-56534dfc2552 service nova] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:53:37.382852 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-f80d2b92-7517-4930-b1a4-56534dfc2552 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:53:37.396429 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d2b92-7517-4930-b1a4-56534dfc2552 service nova] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:53:37.401305 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350ccb5-56b7-4540-b276-2f15801608dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:53:37.401893 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350ccb5-56b7-4540-b276-2f15801608dd service placement] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:53:37.402167 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350ccb5-56b7-4540-b276-2f15801608dd service placement] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:53:37.402434 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350ccb5-56b7-4540-b276-2f15801608dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:53:37.404047 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7350ccb5-56b7-4540-b276-2f15801608dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:53:37.415666 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350ccb5-56b7-4540-b276-2f15801608dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:53:37.417252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350ccb5-56b7-4540-b276-2f15801608dd service placement] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:53:37.427504 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1308/2602] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:53:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:53:47.422627 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=6 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:54:29.173702 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f549ed-c53b-49e2-8dbd-13f288187a0c None None] REQUEST_METHOD: `POST` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:54:29.174255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f549ed-c53b-49e2-8dbd-13f288187a0c None None] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:54:29.174255 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f549ed-c53b-49e2-8dbd-13f288187a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:54:29.193740 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.auth.core [None req-97f549ed-c53b-49e2-8dbd-13f288187a0c None None] MFA Rules not processed for user `b69c66f5c4c34e0eb85975cd7d53a90c`. Rule list: `[]` (Enabled: `True`). {{(pid=81701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 18 11:54:29.194976 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-97f549ed-c53b-49e2-8dbd-13f288187a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:54:29.201806 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1295/2603] 10.222.0.17 () {58 vars in 958 bytes} [Tue Oct 18 11:54:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 18 11:54:39.188941 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=3 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:55:29.332939 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350ccb5-56b7-4540-b276-2f15801608dd service placement] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:55:29.334338 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-7350ccb5-56b7-4540-b276-2f15801608dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:55:29.389222 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350ccb5-56b7-4540-b276-2f15801608dd service placement] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:55:29.392764 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d84ad-45a1-419a-8385-95219ab25ed0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb051b00a16949798158bac1ac57d36f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:55:29.393391 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d84ad-45a1-419a-8385-95219ab25ed0 service cinder] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:55:29.393665 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d84ad-45a1-419a-8385-95219ab25ed0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:55:29.393930 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d84ad-45a1-419a-8385-95219ab25ed0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:55:29.395612 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-387d84ad-45a1-419a-8385-95219ab25ed0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:55:29.406575 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d84ad-45a1-419a-8385-95219ab25ed0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:55:29.407984 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d84ad-45a1-419a-8385-95219ab25ed0 service cinder] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:55:29.418134 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1309/2604] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:55:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:55:32.732513 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f549ed-c53b-49e2-8dbd-13f288187a0c None None] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:55:32.733979 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-97f549ed-c53b-49e2-8dbd-13f288187a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:55:32.746477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f549ed-c53b-49e2-8dbd-13f288187a0c None None] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:55:32.750261 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf19fa2-ad39-4496-bd01-2100426e04d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:55:32.750864 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf19fa2-ad39-4496-bd01-2100426e04d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:55:32.751139 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf19fa2-ad39-4496-bd01-2100426e04d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:55:32.751404 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf19fa2-ad39-4496-bd01-2100426e04d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:55:32.753161 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-2bf19fa2-ad39-4496-bd01-2100426e04d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:55:32.767410 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf19fa2-ad39-4496-bd01-2100426e04d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:55:32.768823 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf19fa2-ad39-4496-bd01-2100426e04d2 service neutron] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:55:32.779767 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1296/2605] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:55:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:55:38.026198 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d84ad-45a1-419a-8385-95219ab25ed0 service cinder] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:55:38.036334 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d84ad-45a1-419a-8385-95219ab25ed0 service cinder] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:55:38.040087 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162ff74-180e-4e43-95e9-cff0962f35ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd301f3d9aeb644a38c31dbdbef2a48c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:55:38.040713 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ff74-180e-4e43-95e9-cff0962f35ff service neutron] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:55:38.041043 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ff74-180e-4e43-95e9-cff0962f35ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:55:38.041312 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ff74-180e-4e43-95e9-cff0962f35ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:55:38.043701 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0162ff74-180e-4e43-95e9-cff0962f35ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:55:38.109856 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162ff74-180e-4e43-95e9-cff0962f35ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:55:38.111285 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162ff74-180e-4e43-95e9-cff0962f35ff service neutron] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:55:38.122632 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1310/2606] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:55:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:55:41.185327 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf19fa2-ad39-4496-bd01-2100426e04d2 service neutron] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:55:41.198166 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf19fa2-ad39-4496-bd01-2100426e04d2 service neutron] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:55:41.204590 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafe9fe-6fbc-4fea-b929-64029118ba86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:55:41.205229 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafe9fe-6fbc-4fea-b929-64029118ba86 service placement] REQUEST_METHOD: `GET` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:55:41.205533 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafe9fe-6fbc-4fea-b929-64029118ba86 service placement] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:55:41.205798 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafe9fe-6fbc-4fea-b929-64029118ba86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:55:41.207523 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-0fafe9fe-6fbc-4fea-b929-64029118ba86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:55:41.219439 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafe9fe-6fbc-4fea-b929-64029118ba86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:55:41.221051 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafe9fe-6fbc-4fea-b929-64029118ba86 service placement] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:55:41.232257 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1297/2607] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:55:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:55:44.399978 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162ff74-180e-4e43-95e9-cff0962f35ff service neutron] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:55:44.401785 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-0162ff74-180e-4e43-95e9-cff0962f35ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:55:44.415095 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162ff74-180e-4e43-95e9-cff0962f35ff service neutron] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:55:44.420685 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:55:44.422020 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:55:44.422020 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:55:44.422020 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] PATH_INFO: `/v3/users/b69c66f5c4c34e0eb85975cd7d53a90c` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:55:44.423273 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69c66f5c4c34e0eb85975cd7d53a90c)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:55:44.425220 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:55:44.473681 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69c66f5c4c34e0eb85975cd7d53a90c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.486194 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69c66f5c4c34e0eb85975cd7d53a90c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.486847 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69c66f5c4c34e0eb85975cd7d53a90c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.493750 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69c66f5c4c34e0eb85975cd7d53a90c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.494405 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69c66f5c4c34e0eb85975cd7d53a90c'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.507451 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1311/2608] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:55:44 2022] DELETE /identity/v3/users/b69c66f5c4c34e0eb85975cd7d53a90c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:55:44.678447 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafe9fe-6fbc-4fea-b929-64029118ba86 service placement] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:55:44.688907 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafe9fe-6fbc-4fea-b929-64029118ba86 service placement] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:55:44.745903 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:55:44.746458 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:55:44.746737 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:55:44.747009 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] PATH_INFO: `/v3/projects/b6ef293a199241bf94848b937058ced5` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:55:44.748105 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6ef293a199241bf94848b937058ced5)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:55:44.750039 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:55:44.795607 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6ef293a199241bf94848b937058ced5', 'request_id': 'req-508065b7-1ae4-45c0-b746-7fe565a366d9'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.801085 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6ef293a199241bf94848b937058ced5', 'request_id': 'req-508065b7-1ae4-45c0-b746-7fe565a366d9'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.808058 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6ef293a199241bf94848b937058ced5', 'request_id': 'req-508065b7-1ae4-45c0-b746-7fe565a366d9'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.818956 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] The token cache is being invalidate because project b6ef293a199241bf94848b937058ced5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:55:44.819245 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.819885 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.821392 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1298/2609] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:55:44 2022] DELETE /identity/v3/projects/b6ef293a199241bf94848b937058ced5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:55:44.825565 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:55:44.826990 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:55:44.842190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f136c0-333f-475e-bb5e-b1ed400d1fa3 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:55:44.846803 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:55:44.898450 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:55:44.899072 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:55:44.899348 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:55:44.899615 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] PATH_INFO: `/v3/users/ccac288563cb4db8ade7094e6b95f61e` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:55:44.900725 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccac288563cb4db8ade7094e6b95f61e)` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:55:44.903141 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:55:44.947987 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccac288563cb4db8ade7094e6b95f61e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.960705 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccac288563cb4db8ade7094e6b95f61e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.961414 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccac288563cb4db8ade7094e6b95f61e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.969005 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccac288563cb4db8ade7094e6b95f61e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.969829 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.notifications [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccac288563cb4db8ade7094e6b95f61e'} {{(pid=81700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:44.982707 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1312/2610] 10.222.0.17 () {58 vars in 1224 bytes} [Tue Oct 18 11:55:44 2022] DELETE /identity/v3/users/ccac288563cb4db8ade7094e6b95f61e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:55:45.155117 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] Authenticating user token {{(pid=81701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:55:45.156626 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.fernet_utils [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:55:45.171240 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508065b7-1ae4-45c0-b746-7fe565a366d9 admin admin] Validating token access rules against request {{(pid=81701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:55:45.228531 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5f193bd980d483b8200f674f5249745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c023e59351c4a06998c34820bba9111', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:55:45.229197 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:55:45.229477 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:55:45.229751 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] PATH_INFO: `/v3/projects/56c1847e28f3482d8a2eeecfd5f32806` {{(pid=81701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:55:45.230860 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56c1847e28f3482d8a2eeecfd5f32806)` {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:55:45.232754 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] RBAC: Authorization granted {{(pid=81701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:55:45.238793 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:55:45.282424 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56c1847e28f3482d8a2eeecfd5f32806', 'request_id': 'req-82507b3e-1846-4a8c-8515-edcae2b24e77'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:45.287747 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56c1847e28f3482d8a2eeecfd5f32806', 'request_id': 'req-82507b3e-1846-4a8c-8515-edcae2b24e77'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:45.294890 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56c1847e28f3482d8a2eeecfd5f32806', 'request_id': 'req-82507b3e-1846-4a8c-8515-edcae2b24e77'} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:45.305959 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] The token cache is being invalidate because project 56c1847e28f3482d8a2eeecfd5f32806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 18 11:55:45.306279 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:45.306905 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG keystone.notifications [None req-82507b3e-1846-4a8c-8515-edcae2b24e77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 18 11:55:45.308401 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: [pid: 81701|app: 0|req: 1299/2611] 10.222.0.17 () {58 vars in 1233 bytes} [Tue Oct 18 11:55:45 2022] DELETE /identity/v3/projects/56c1847e28f3482d8a2eeecfd5f32806 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 18 11:55:54.979190 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:55:55.303022 dv-cinder-815582-17-iscsi devstack@keystone.service[81701]: DEBUG dbcounter [-] [81701] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 18 11:58:40.743397 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] Authenticating user token {{(pid=81700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 18 11:58:40.744830 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:58:40.887252 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71635185-53b0-422b-9190-ec59dfa9d933 admin admin] Validating token access rules against request {{(pid=81700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 18 11:58:40.945993 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20b8e0-b559-4d76-a881-75422dbc33d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99ec4a4276f345c2998723e554ab9608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4daff69874da475b99a7c445df7b3f4d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 18 11:58:40.946601 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20b8e0-b559-4d76-a881-75422dbc33d3 service placement] REQUEST_METHOD: `GET` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 18 11:58:40.946883 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20b8e0-b559-4d76-a881-75422dbc33d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 18 11:58:40.947157 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20b8e0-b559-4d76-a881-75422dbc33d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 18 11:58:40.948885 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.fernet_utils [None req-1e20b8e0-b559-4d76-a881-75422dbc33d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 18 11:58:40.960116 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20b8e0-b559-4d76-a881-75422dbc33d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 18 11:58:40.961670 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20b8e0-b559-4d76-a881-75422dbc33d3 service placement] RBAC: Authorization granted {{(pid=81700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 18 11:58:40.971449 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: [pid: 81700|app: 0|req: 1313/2612] 10.222.0.17 () {60 vars in 1365 bytes} [Tue Oct 18 11:58:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 18 11:58:50.967231 dv-cinder-815582-17-iscsi devstack@keystone.service[81700]: DEBUG dbcounter [-] [81700] Writing DB stats keystone:SELECT=89 {{(pid=81700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}